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SUMMARY 

This  report  documents  the  effort  to  identify  a  candidate  numerical  weather  prediction  model,  suitable  for 
theater  use,  from  currently  available  off-the-shelf  mesoscale  models.  The  candidate  models  were  the 
Mesoscale  Model  Version  5  (MM5),  the  Navy  Operational  Regional  Atmospheric  Prediction  System 
Version  6  (NORAPS6),  and  the  Regional  Atmospheric  Modeling  System  (RAMS).  The  Air  Force  (AF) 
regional  forecast  model,  the  Relocatable  Window  Model  (RWM)  was  the  baseline  against  which  the 
other  models  were  evaluated.  Based  on  the  model  comparison,  DSWA  chose  to  use  RAMS  to  provide 
high  resolution  meteorological  forecasts  for  its  transport  and  diffusion  program.  The  report  also 
describes  the  subsequent  adaptation  and  application  of  this  model  to  actual  field  tests.  This  report  is 
functionally  divided  into  three  parts  to  document  this  process. 

The  first  part  of  the  report  (Sections  1-5)  describes,  in  detail,  the  model  comparison  process.  It  begins 
with  a  description  of  the  requirements  that  were  used  to  develop  the  comparison  criteria.  As  the  forecast 
model  was  to  be  used  in  the  field  this  consisted  of  a  set  of  parameters  such  as  run  time  goals,  grid  size, 
and  spacing  that  were  to  be  used.  In  addition,  a  number  of  test  cases  were  defined  to  exercise  the  models 
in  a  number  of  different  situations.  Specifically,  the  model  comparison  included  two  data  sets  in  Alaska, 
Central  America,  the  Middle  East,  and  Korea.  The  initial  comparison  case  was  located  in  the 
Continental  United  States  (CONUS).  The  meteorological  data  sets  were  obtained  from  the  National 
Center  for  Atmospheric  Research  (NCAR)  and  from  the  Air  Force  Global  Weather  Central  (AFGWC). 
The  comparison  of  the  performance  of  the  models  was  accomplished  through  a  number  of  statistical 
measures.  The  ranking  of  the  models  was  consistent.  RAMS  was  first,  followed  closely  by  MM5.  - 

The  next  part  (Section  6)  reports  on  efforts  to  adapt  the  RAMS  model  to  operational  forecasting.  The 
primary  goal  of  the  effort  described  in  this  portion  of  the  report  was  to  reduce  the  execution  time 
required  for  a  given  forecast.  Major  reductions  in  run-time  were  accomplished  by  using  the  new  parallel 
computer  systems  such  as  the  IBM  SP-2  or  HP  Exemplar.  This  adaptation  built  on  earlier  work  using 
parallel  clusters  of  independent  workstations  and  resulted  in  significant  run-time  reductions.  In  addition, 
the  possibility  of  using  a  technique  called  the  Fully  Equivalent  Operational  Model  (FEOM)  was  explored 
as  a  potential  means  of  improving  model  performance  without  laborious  recoding.  The  proof  of  concept 
used  the  radiative  transport  module  from  RAMS  in  one  configuration.  The  FEOM  equivalent  ran  a 
thousand  times  faster  with  an  accuracy  difference  of  less  than  one  tenth  of  a  percent.  Finally,  this 
section  describes  the  exploration  of  a  new  scheme  for  computation  of  gradients  in  models  using  a  terrain 
following  coordinate  system.  The  use  of  this  code  improved  the  forecast  of  plume  dispersion  from  the 
test  at  WSMR. 

The  final  part  (Section  7)  documents  the  use  of  RAMS  during  numerous  field  tests  at  White  Sands 
Missile  Range  in  support  of  the  transport  and  diffusion  tests.  The  support  for  these  tests  resulted  in  a 
better  user  interface  and  the  integration  of  RAMS  output  with  DSWA’s  SCIPUFF  dispersion  model. 
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SECTION  1 

MODEL  COMPARISON 

1.1  INTRODUCTION  AND  OVERVIEW. 

Air  Force  Weather  (AFW)  identified  a  deficiency  in  producing  timely  and  accurate  theater  forecasts.  To 
overcome  this  deficiency,  AFW  and  the  Defense  Special  Weapons  Agency  (DSWA),  formerly  the 
Defense  Nuclear  Agency  (DNA),  sponsored  the  development  of  the  Theater  Forecast  Model  (TPM). 
There  were  two  main  objectives  of  the  TFM  development  effort.  The  first  objective  was  to  identify  a 
candidate  numerical  weather  prediction  model,  suitable  for  theater  use,  from  currently  available  models. 
Once  the  best  model  was  selected,  it  was  to  be  adapted  to  meet  the  needs  of  deployed  forces  on  the 
battlefield. 

The  actual  model  selection  process  began  with  Air  Weather  Service’s  (AWS)  “Combat  Weather  System 
Technical  Alternatives  Study.”  This  study,  conducted  by  Dynamic  Research  Corporation  in  1993, 
reviewed  over  100  different  numerical  weather  prediction  models,  and  compared  and  ranked  their 
capabilities.  The  ranking  was  based  on  an  integrated  set  of  customer  requirements  and  relied  on 
published  and  developer  provided  information  on  the  models. 

The  second  phase  of  the  selection  process  was  the  detailed  performance  comparisons  of  three  of  the  top 
ranked  models.  Performance  in  this  phase  of  the  evaluation  was  defined  as  how  well  the  model  could 
forecast  the  observed  weather.  The  emphasis  of  the  performance  evaluation  was  on  statistical  measures 
of  model  performance  rather  than  subjective  measures.  The  ability  of  the  models  to  operate  on  a 
workstation  was  also  evaluated  by  actually  making  forecasts  on  a  representative  workstation.  This 
performance  evaluation  is  documented  in  section  one  of  this  report,  and  was  extracted  from  an  informal 
report  prepared  to  support  the  TFM  decision  (Bauer  et  al.,  1995). 

The  baseline  was  the  performance  of  the  current  AF  model  operating  at  Air  Force  Global  Weather 
Central  (AFGWC),  the  Relocatable  Window  Model  (RWM).  The  three  candidates  are  the  Pennsylvania 
State  University  (PSU)ANational  Center  for  Atmospheric  Research  (NCAR)  Mesoscale  Model  Version  5 
(MM5),  the  Navy  Operational  Regional  Atmospheric  Prediction  System  Version  6  (NORAPS6)  model, 
and  the  Colorado  State  University  (CSU)  Regional  Atmospheric  Modeling  System  (RAMS).  To  insure 
the  model  evaluation  was  not  biased  to  a  particular  region  or  climate  type,  test  forecasts  were 
accomplished  in  five  regions  of  the  world.  These  regions  were  the  Continental  United  States  (CONUS), 
Alaska,  Korea,  Central  America,  and  the  Middle  East 

1.2  OVERVIEW. 

A  previous  comparison  study  focused  on  the  ability  of  mesoscale  prediction  models  to  make  accurate 
forecasts  (Pielke  and  Pearce,  1994).  As  noted  in  that  study  “modelers  were  free  to  decide  their  own 
initialization  and  data  assimilation  procedures  so  that,  inevitably,  the  results  represent  differing  levels  of 
sophistication  of  external  forcing.  Some  modelers  were  able  to  carry  out  a  series  of  experiments  to 
determine  the  most  appropriate  values  of  some  assigned  model  parameters  . . .”  (Busch  et  al.,  1994). 

The  guiding  principal  behind  the  current  study  was  to  perform  a  pseudo-operational  comparison  under 
standard  conditions  using  identical  data.  Expected  field  operational  constraints  on  forecast  area, 
resolution,  and  data  available  were  imposed.  Once  the  models  were  set  up,  no  changes  were  made  in 
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their  configuration,  and  they  remained  the  same  for  all  forecasts,  independent  of  the  geographical  region. 
The  models  used  were  obtained  from  the  sponsors  of  the  models  and  reflect  the  “standard”  model 
available  at  the  time  the  study  began.  Since  many  of  the  parameterizations  used  in  the  models  were  scale 
dependent,  careful  selection  of  model  options  was  required.  The  models’  authors,  to  insure  that  the 
correct  options  had  been  selected,  reviewed  these  selections  and  the  output  from  the  first  case.  Two  of 
the  models,  MM5  and  RAMS,  were  under  active  development  and  new  algorithms  and  capabilities  were 
being  added  as  they  were  completed  and  tested.  Since  the  focus  of  this  study  was  to  operate  the  models 
as  if  they  were  operational,  no  attempt  was  made  to  access  these  research  capabilities  during  the 
evaluation  of  the  models.  The  infusion  of  new  technology  into  operations  would  require  further 
extensive  testing  and  evaluation. 

1.3  REQUIREMENTS. 

The  Combat  Weather  System  (CWS)  development  was  cancelled  by  the  Air  Force  (AF)  shortly  after  this 
effort  was  initiated.  However,  AWS  reaffirmed  the  requirement  for  a  TFM  model  capable  of  providing 
theater  forecasts  in  the  field.  The  requirements  for  this  model  were  those  developed  for  the  CWS 
forecast  module  plus  additional  requirements  that  reflected  AFGWC’s  role  in  producing  theater 
forecasts.  Section  2  reviews  all  the  requirements  for  the  forecast  model  and  develops  an  integrated  set  of 
requirements. 

1.4  TEST  PROCESS. 

The  models  were  configured  to  use  the  “best”  parametric  options  available,  to  use  identical  sets  of  input 
data,  and  to  generate  forecasts  that  could  be  directly  compared.  The  requirements  provided  a 
challenging  set  of  criteria  to  evaluate  in  a  single  program. 

To  meet  the  AF  requirements,  each  model  must  ultimately  be  able  to  operate  correctly  on  a  workstation. 
However,  most  of  the  models  were  developed  on  some  version  of  a  mainframe  CRAY  computer  and 
would  operate  on  that  platform  with  minimal  changes.  After  obtaining  the  models,  we  attempted  to 
install  all  but  RWM  on  DSWA’s  UNIX  based  CRAY  and  an  RS/6000  (Figure  1-1.).  RWM  normally 
operates  on  a  CRAY  XMP  with  data  input  support  from  the  mainframe  computers  and  databases  at 
AJFGWC.  The  model  had  been  ported  and  validated  on  a  Unix  RS/6000  workstation  at  AFGWC  and 
this  version  was  used.  RWM  was  run  only  on  the  workstation  since  no  validated  version  for  a  Unix- 
based  CRAY  exists.  A  convenient  and  available  workstation,  the  IBM  RS/6000,  was  used  as  the  test 
workstation  for  all  forecasts.  The  RS/6000  is  a  RISC-based  machine  made  by  IBM  that  is  in  the  middle- 
to  high-performance  category  today,  but  will  probably  represent  the  medium-  to  low-end  capability  when 
the  Theater  Forecast  Model  (TFM)  is  operational.  The  configuration  of  models  and  input  data  sets  were 
the  same  on  both  the  workstation  and  the  CRAY. 

The  models,  options  selected,  and  their  run  performance  will  be  described,  in  detail,  in  Section  3.  Each 
model  has  one  or  more  data  ingestion  programs  or  routines  associated  with  it.  These  ingestion  routines 
often  manipulate  the  data  and  boundary  conditions  to  reduce  startup  generated  “noise.”  For  this  reason, 
each  model’s  native  data  ingestion  routine  was  used  to  construct  the  initial  forecast  fields  from  the 
observations  and  boundary  conditions  supplied  by  the  global  circulation  models.  Since  data  from  both 
National  Center  for  Atmospheric  Research  (NCAR)  and  AFGWC  were  used  to  initialize  the  various 
forecast  runs,  we  translated  the  data  into  a  standard  data  format  and  modified  each  data  ingestion  routine 
to  accept  the  standard  format. 
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Figure  1-1.  Initial  steps. 


The  basic  forecast  modules  were  used,  to  the  extent  possible,  as  received  (Figure  1-2.).  The  only  - 
changes  made  were  to  read  the  standard  data  format  and  to  remove  any  smoothing  of  the  output.  The 
elimination  of  smoothing  was  done  to  insure  that  the  forecast  ability  and  not  display  capability  was 
measured.  Each  test  consisted  of  an  independent  36  hour  forecast.  This  forecast  period  was  selected  to 
reduce  the  amount  of  run  time  required  for  testing  while  allowing  the  models  adequate  time  to  “spin-up.” 
“Spin-up”  refers  to  the  time  required  for  the  model  to  overcome  “noise”  resulting  from  the  initial 
conditions.  For  the  basic  or  benchmark  tests,  data  was  obtained  for  a  three-day  period. 


Figure  1-2.  Forecast  flow. 


Once  a  forecast  was  accomplished,  the  computed  data  was  interpolated  to  the  observation  points. 
Forecast  data  was  compared  to  observed  data  using  a  program  called  REVU,  which  we  designed  to 
produce  the  required  statistical  data.  This  program  and  the  quantities  it  compares  are  discussed  in 
Section  3.6.  Figure  1-3  indicates  that  we  accomplished  two  independent  forecasts  during  each  three  day 
period.  One  forecast  was  initialized  at  the  beginning  of  the  three  days  and  the  second  initialized  midway 
through  the  data  period.  Forecasts  were  made  at  the  times  indicated  to  allow  comparison  with  surface 
and  upper  air  observations. 


Figure  1-3.  Forecast  periods  and  production  times. 

The  test  series  focused  on  the  ability  of  the  models  to  perform  at  the  minimum  or  threshold  TFM  leyel. 
Two  excursions  were  also  examined.  First,  the  model’s  performance  was  evaluated  in  a  theater  where 
portions  of  the  observations  were  not  available,  perhaps  due  to  enemy  actions  or  simple  communications 
failure.  Second,  the  potential  improvements  due  to  the  higher  resolution  inherent  in  the  objective  TFM 
requirements  were  examined. 

1 .4. 1  Threshold  Forecasts. 

Table  1-1  summarizes  the  threshold  tests.  We  began  by  running  a  well  reported  set  of  data 
corresponding  to  the  Atmospheric  Variability  Experiment — Severe  Environmental  Storms  and 
Mesoscale  Experiment  (SESAME)  1  Test  which  took  place  in  CONUS  during  April  10-11, 1979. 

Our  test  forecasts  were  based  on  and  compared  to  data  extracted  from  NCAR  archives  corresponding  to 
the  days  April  9-1 1.  With  the  RWM  exception  noted,  we  intended  to  duplicate  the  forecasts  on  the 
CRAY  and  the  workstation.  This  exercise  was  to  provide  specific  information  about  code  portability, 
any  performance  degradation  on  a  workstation,  and  the  integrity  of  each  model — whether  it  produces  the 
same  results  on  the  workstation  as  on  the  supercomputer.  We  were  able  to  accomplish  this  for  all  the 
candidate  models  except  MM5.  Each  of  the  models  that  were  able  to  run  on  both  platforms  produced 
forecasts  that  were  independent  of  the  platform  used.  We  were  unable  to  get  the  original" version  of 
MM5  to  operate  on  the  workstation.  This  was  entirely  due  to  problems  encountered  with  the  data 
ingestion  routines.  We  obtained  what  was  described  as  a  workstation  version,  but  found  that 
workstation  compatible  data  ingestion  routines  do  not  exist  for  MM5.  The  workstation  version  of  MM5 
exhibited  a  large  number  of  changes  from  the  CRAY  version.  Rewriting  the  ingestion  routines  was 
started  but  was  quickly  determined  to  be  beyond  the  scope  of  this  effort.  A  hybrid  approach  was 
considered,  e.g.  running  the  data  ingestion  routines  on  the  CRAY  and  transferring  their  output  to  the 
workstation.  However,  MM5  uses  binary  transfer  files,  and  data  conversion  and  validation  would  have 
significantly  delayed  testing.  As  a  result  of  these  problems,  we  restricted  MM5  to  the  CRAY.  The 
workstation  forecast  module  was  tested  on  the  CRAY  by  comparing  its  forecast  to  that  produced  by  the 
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CRAY  version  using  identical  input  data  created  by  the  CRAY  ingestion  routines.  Even  with  the 
substantive  rewrite  evident  in  the  workstation  version,  both  versions  gave  the  same  results.  The  data  for 
SESAME  are  presented  in  Section  4.1.1. 


Table  1-1.  Threshold  forecast  cases. 


Test 

Benchmark 

Benchmark 

Benchmark 

Data  Denied 

Data  Source 

NCAR 
April  79 

NCAR 
August  94 

AFGWC 
November  94 

NCAR 
September  93 

Theater 

CONUS 

SESAME 

Alaska 

Summer 

Winter 

Central  America 

Wet 

Winter 

Middle  East 

Summer 

Winter 

Korea 

Monsoon 

Winter 

Fall 

Once  we  completed  the  SESAME  comparisons,  we  began  the  remaining  benchmark  tests.  SESAME 
allowed  comparison  of  the  model’s  performance  in  the  CONUS  and  between  platforms.  The  next  tests 
used  August  1994  data  from  NCAR  but  were  accomplished  for  Alaska,  Korea,  Central  America,  and  the 
Middle  East.  The  final  tests  in  this  series  looked  at  simulated  operational  data  from  AFGWC.  AFGWC 
personnel  downloaded  this  data  in  November  1994.  The  data  corresponds  to  the  operational  time 
AFGWC  would  have  initialized  the  TFM.  The  intent  was  to  reproduce  the  environment  in  which  the 
TFM  would  need  to  operate.  Late  reporting  stations  were  not  included.  In  all  of  these  post-SESAME 
tests,  NORAPS6  and  MM5  were  used  on  the  CRAY,  and  RWM  and  RAMS  on  the  workstation.  All  of 
the  data  from  the  August  tests  are  discussed  in  detail  in  Section  4.1.2  and  the  November  tests  in  Section 
4.1.3. 

1.4.2  Data  Denied  Forecast. 

In  addition  to  basic  threshold  forecasts,  each  model’s  ability  to  produce  realistic  forecasts  in  areas  where 
data  is  denied  needed  to  be  accomplished.  These  were  conducted  using  surface  and  upper  air  data 
obtained  from  NCAR  for  September  1993  in  the  Korean  theater.  Three  sets  of  forecasts  were  produced. 
In  the  first  set,  a  basis  of  comparison  was  established  and  all  available  data  was  used  to  initialize  the 
predictions.  In  the  second  set,  only  a  single  station  (at  Seoul,  assumed  to  be  the  location  of  the  TFM) 
was  used.  This  situation  might  occur  on  a  battlefield  due  to  enemy  action  or  to  a  communications  failure 
caused  by  any  number  of  factors.  The  third  forecast  was  accomplished  with  the  surface  and  upper  air 
data  from  China  and  North  Korea  eliminated.  Military  operations  will  not  cease  and  there  is  a  need  to 
provide  a  credible  forecast  in  this  situation.  The  single  station  initialization  results  were  of  minimal 
value  for  all  models.  In  the  case  where  stations  in  North  Korea  and  China  were  eliminated,  all  the 
models  did  surprisingly  well.  This  is  discussed  in  more  detail  in  Section  4.2. 


5 


•  Figure  1-4.  Data  Denied  Flow. 


1 .4.3  Objective  Requirements  and  Enhanced  Forecast. 

The  AF  has  established  a  set  of  objective  requirements  that  they  ultimately  want  to  meet.  These 
conditions  correspond  to  an  enhanced  resolution,  with  grid  size  reduced  from  25  nmi  to  6  nmi.  This 
increased  resolution  will  emphasize  non-hydrostatic  terms  in  physical  models  and  parameterizations. 
RAMS  and  MM5  both  include  options  to  use  non-hydrostatic  equations,  whereas  NORAPS6  can  only 
use  a  hydrostatic  approximation.  This  difference  will  have  a  bearing  on  code  selection,  as  the  model 
should  be  viable  at  the  objective  resolution  to  provide  for  required  future  growth.  It  could  also  be 
required  to  support  a  cloud  model  that  might  supplement  the  theater  forecast  model.  We  attempted  to 
replicate  the  Central  American  and  Middle  East  theater  forecasts  with  these  tests  using  the  nested  grid 
option  where  available.  MM5,  NORAPS6,  and  RAMS  claim  to  be  able  to  develop  the  smaller  more 
finely  resolved  grids  that  interact  with  a  larger  more  coarsely  resolved  grid.  There  is  no  AF  requirement 
for  an  ability  to  generate  these  nested  grids  and  the  capability  was  used  here  only  to  look  at  the  effect  of 
enhanced  resolution.  Central  America  and  the  Middle  East  provide  an  interesting  contrast  for  this 
evaluation.  The  increased  resolution  was  expected  to  improve  the  forecast  for  Central  America  but  have 
less  impact  on  the  Middle  East.  Again,  the  data  ingestion  routines  for  MM5  proved  to  be  a  formidable 
obstacle.  We  were  unable  to  initialize  these  forecasts  for  MM5.  Comparisons  for  NORAPS6  and 
RAMS  are  provided  in  Section  4.3. 
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Figure  1-5.  Objective  Test  Flow. 
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SECTION  2 
TEST  CRITERIA 


This  section  described  the  test  criteria  that  were  used  in  evaluating  the  models.  The  test  criteria  were 
derived  from  diverse  Department  of  Defense  and  AF  requirements  defined  at  the  start  of  the  study.  In 
addition,  requirements  were  developed  jointly  by  DSWA  and  the  AF  in  anticipation  of  changes  in  the 
published  requirements.  As  these  requirements  have  continued  to  evolve  since  this  study  began,  this 
section  documents  the  requirement  baseline  at  the  time  the  study  was  conducted.  The  following 
describes  and  traces  these  requirements  and  the  resulting  test  criteria. 

2.1  MODEL  CAPABILITIES  VS.  WEATHER  FORECASTING  AND  OBSERVING  CAPABILITIES. 

Hydrodynamic  models  attempt  to  describe  the  future  state  of  the  atmosphere  from  a  set  of  initial 
conditions.  As  computational  capabilities  have  improved,  models  have  improved  their  resolution,  model 
physics,  and  ability  to  use  diverse  input.  Their  goal  has  been  to  improve  temporal  and  spatial  resolution 
of  predicted  weather  features  using  the  best  science  available  within  the  limits  imposed  by 
computational  and  hardware  constraints.  Judging  the  utility  of  a  hydrodynamic  model  based  solely  on 
results  for  a  few  output  parameters  (such  as  wind  speed  and  direction,  temperatures,  pressure  values, 
humidity  within  a  set  range  of  error  bars)  can  be  misleading.  Models  must  be  judged  both  by  statistical 
comparisons  of  forecast  and  observed  parameters  and  by  their  ability  to  produce  coherent  descriptions  of 
the  three-dimensional  structure  of  tropospheric  weather  and  the  global,  synoptic,  and  mesoscale  features 
that  drive  the  weather.  Accuracy  is  desired,  but  a  model’s  forecast  does  not  need  to  be  perfect,  if  its 
biases  can  be  quantified. 

2.2  MODEL  OUTPUTS  VS.  CUSTOMER  REQUIREMENTS. 

Forecast  models  are  usually  judged  by  their  ability  to  forecast  the  development,  location,  temporal  and 
spatial  resolution  of  major  global,  synoptic,  and  mesoscale  features  and  atmospheric  structure,  rather 
than  their  ability  to  predict  specific  parameters.  AWS  customer  requirements  for  weather  support 
typically  are  based  on  go/no-go  criteria  for  a  particular  operation  or  on  the  known  weather  sensitivities  of 
a  particular  weapon  system.  These  criteria  are  presented  in  various  weather  support  documents  as 
Requirement  Correlation  Matrices  (RCMs).  While  useful  for  defining  goals  for  weather  support 
systems,  these  criteria  may  be  unrealistic  forecast  model  goals  because  weather  observing  platforms 
cannot  measure  some  of  these  quantities,  much  less  forecast  them.  For  models,  the  goal  is  to  build  the 
best  atmospheric  model,  so  the  best  possible  estimate  of  specific  required  parameters  will  be  produced. 

2.3  TRACEABILITY  AND  PRIORITY  OF  REQUIREMENTS. 

The  test  criteria  were  traceable  to  weather  support  requirements.  Collectively,  the  test  criteria  ensure 
that  the  model  chosen  met  the  broadest  possible  spectrum  of  weather  support  needs. 

2.4  TYPES  OF  REQUIREMENTS. 

In  establishing  reasonable  test  criteria,  we  considered  various  types  of  requirements  and  several  different 
requirement  sources.  We  also  considered  both  objective  and  subjective  model  evaluation  criteria 

2.4.1  Objective  Evaluation  Criteria. 

Objective  test  criteria  compare  a  model’s  output  parameters  and  characteristics  to  a  desired  standard. 
Each  standard  is  derived  from  or  traceable  to  a  user  requirement  or  quantifiable  test  criteria  based  on  the 
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ORD,  the  SOW,  or  other  meteorological  support  studies.  Examples  of  objective  test  criteria  are  the 
model  verification  statistics  produced  by  the  Mission  Research  Corporation  (MRC)-ASTeR  model 
comparison  software,  REVU  that  is  described  in  Section  3.6.  Verification  statistics  produced  by  each 
candidate  model  are  cross-compared  to  identify  which  model  produces  the  best  estimate  of  various 
atmospheric  parameters  as  a  function  of  forecast  lead-time.  Key  subclasses  of  quantitative  test  criteria 
are  discussed  in  the  following  paragraphs. 

2.4. 1.1  Meteorological  Parameters.  The  primary  atmospheric  parameters  output  by  dynamic  models  in 
the  United  States  Air  Force  (USAF)  Electronic  System  Center  (ESC)  study  (4-2)  are:  wind  components 
(u,  v,  w),  potential  temperature,  specific  humidity  (q),  and  pressure.  These  parameters  were  judged 
using  the  data  generated  by  the  REVU  software  that  compared  model  output  to  the  RAQBS  and  surface 
data. 


2.4. 1.2  Customer  Support  Requirements.  Many  of  the  applications  and  weather  support  requirements 
for  the  TFM  are  summarized  in  RCM  matrices  in  the  ORD,  the  Weather  2000  Study,  and  a  previous 
study.  Combat  Weather  Systems  Technical  Alternative  Study  Interim  Report,  by  ESC  in  1993.  While 
these  specify  the  accuracy  of  meteorological  parameters  that  are  required  to  support  a  mission  or  weapon 
system,  many  of  these  stated  values  were  really  requirements  for  meteorological  observations,  and  were 
not  the  accuracy  required  from  a  tactical  weather  forecast  model. 


2.4.2  Subjective  Evaluation  Criteria. 

Subjective  criteria  involve  assessing  model  performance  or  characteristics  based  on  extensive 
experience.  For  example,  a  model  that  provides  superior  forecast  results  may  not  be  transportable  to  a 
workstation  environment,  or  cannot  be  maintained  without  being  thoroughly  rewritten  in  another 
programming  language.  Other  examples  include  models  that  require  excessive  run-time,  do  not  perform 
well  in  data-sparse  regions,  or  do  not  degrade  gracefully  when  part  of  the  forecast  grid  becomes  data- 
denied.  Subjective  criteria  include  comparisons  to  other  meteorological  information,  which  is  not 
subject  to  statistical  evaluation.  This  includes,  for  example,  the  correct  placement  and  intensity  of  a  low- 
pressure  center.  Subjective  model  selection  criteria  are  as  important  as  objective  criteria  and  must  also 
be  traceable  to  ORD  requirements,  support  requirements,  or  support  studies  such  as  the  Weather  2000 
report. 


2.5  REQUIREMENTS  OVERVIEW. 

This  section  briefly  reviews  all  documented  test  requirements.  These  requirements  were  either  explicit 
or  implied.  Derived  requirements  are  summarized  in  Section  2.6.  The  documents  considered,  shown  in 
Table  2-1,  include  both  primary  and  secondary  sources. 


Table  2-1.  Documents  from  which  requirements  are  derived. 


DOCUMENT 

SECTION 

Operational  Requirements  Document  Version  3 

2.5.1. 

Statement  of  Need,  Tactical  Weather  Observing  System 

2.5.2. 

Statement  of  Need,  Tactical  Forecast  System 

2.5.3. 

DNA  Statement  of  Work,  TFM 

2.5.4. 

DNA  Test  Requirements  Letter 

2.5.5. 

Army  Manual  FM34-81/AFM  105-4 

2.5.6. 

CWS  Technical  Alternative  Interim  Report 

2.5.7. 

2.5.1  Operational  Requirements  Document  (ORD)  Version  3. 

The  ORD  (Operational  Requirements  Document-Combat  Weather  System,  1992)  was  a  key  source 
document  driving  the  need  to  upgrade  weather  support  systems  for  theater  applications  and  to  upgrade 
AFGWC  theater  support  models  to  the  state-of-the-art.  The  ORD  stated  that  the  TFM  must  support 
combat  and  support  forces  from  initial  deployment  through  sustained  theater  operations.  The  ORD 
identified  the  mission  areas  listed  in  Table  2-2  as  needing  improved  mesoscale  modeling  weather  - 
support. 

Table  2-2.  ORD  mission  support  areas. 


Air  Force 

Army 

Counter  Air 

Aviation 

Strategic  Attack 

Air  Defense 

Interdiction 

Close  Combat 

Close  Air  Support 

Land  Combat  Engineering  Support 

Strategic  Airlift 

Special  Operations 

Operational  Support 

Fire  Support 

Base  Operability  and  Defense 

Biological  and  Chemical 

Logistics 

The  goal  of  the  TFM  system  was  to  integrate  highly  capable  automated  weather  observing  and 
forecasting  systems  with  combat  planning  and  execution  systems.  Theaters  forecast  support  must 
integrate  observations,  generate  tailored  forecast  information  quickly,  and  distribute  it  to  the  Theater 
Battle  Management  (TBM)  Command,  Control,  Communication,  Computers,  and  Intelligence  (C4I) 
systems.  The  TFM,  as  the  primary  forecast  function  embedded  within  a  tactical  weather  support  system, 
must  respond  to  single  points  of  failure  both  within  and  outside  the  theater  and  should  degrade  gracefully 
to  a  point  where  observations  only,  or  single  station  analyses  are  produced. 

The  TFM  was  to  produce  the  most  accurate  analysis  and  forecast  fields  that  technology  could  provide. 
The  requirements  for  “forecast  fields”  indicated  that  not  only  were  accurate  analyses  of  synoptic  and 
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mesoscale  weather  features  required  but  gridded  fields  of  specific  weather  parameters  were  also 
required.  Many  forecast  functions  at  AFGWC  require  gridded  analysis  fields  of  multiple  parameters  at 
multiple  mesh  resolutions.  The  TFM  needed  to  meet  this  requirement  if  it  was  to  replace  the  RWM  at 
AFGWC.  The  TFM,  as  part  of  a  tactical  weather  support  center,  was  to  provide  a  theater-scale  analysis 
and  forecast  model  capability.  This  use  made  model  run-time  and  user  interface  a  key  selection 
criterion. 

Under  the  description  of  existing  weather  support  components,  the  ORD  described  the  Combat  Air 
Forces  Weather  Software  Package  (CAFWSP)  as  one  of  the  key  interfaces  to  produce  weather  support 
products  for  the  C4I  system.  The  model  selected  as  the  TFM  would  have  generated  output  and  analysis 
fields  that  were  compatible  with  the  CAFWSP. 

Additional  specific  operational  requirements  for  the  TFM  are  listed  starting  on  page  17  of  the  ORD: 

•  The  TFM  must  use  standard  terrain  databases,  AFGWC  supplied  boundary  values,  climatological 
data  bases,  and  theater  weather  observations  (here  the  term  “theater”  was  presumed  to  be  synoptic 
scale  to  mesoscale). 

•  The  Model  must  execute  a  theater  scale  weather  analysis  and  forecast  that  cover  an  area  at  least  . 
1500  nmi  by  1500  nmi  with  an  objective  coverage  area  of  3200  by  3200  nmi. 

•  The  theater  scale  weather  analysis  and  forecast  model  should  have  forecast  outputs  every  3  hours  out 
to  72  hours. 

•  Model  run-time  for  the  above  requirements  should  be  1  hour  or  less.  The  model  should  produce 
uniform  gridded  data  fields  of  the  same  parameters  forecast  by  the  comparable  model  used  at 
AFGWC  (RWM).  Examples  of  these  parameters  are:  temperature,  pressure,  u,v,w,  relative  humidity 
and  specific  humidity. 

•  Based  on  available  data,  the  model  should  gracefully  degrade  to  produce  the  best  possible  analysis 
and  forecast  fields  as  data  becomes  sparse  or  denied. 

•  Horizontal  resolution  must  be  25  nmi  with  an  objective  of  6  nmi  to  produce  databases  and  inputs  to 
support  requirements  such  as  tactical  decision  aids  and  support  to  precision  guided  munitions,  as 
well  as  CBW  and  fallout  predictions. 

•  Software  engineering  requirements  are  discussed  on  page  22  of  the  ORD.  They  are  summarized  as 
follows: 

(1)  The  TFM  must  be  developed  in  an  open  systems  environment  to  enhance  portability  and 
interoperability; 

(2)  Existing  DOD  and  COTS  software  shall  be  used  to  the  extent  possible; 

(3)  Reasonable  software  configuration  management  methods  and  tools  shall  be  used;  and 

(4)  The  system  shall  interface  with  existing  AFGWC  and  C4I  systems. 

The  ORD  also  summarizes  key  model  requirements  in  a  Requirements  Correlation  Matrix  (RCM)  that 
should  be  considered  in  development  of  the  model  selection  criteria. 

2.5.2  Statement  of  Need  (SON)  for  Tactical  Weather  Observing  Systems  (TWOS)  (USAF  21 1-89). 

This  document  was  a  SON  for  weather  observing  systems,  not  for  the  forecast  system.  It  stated 
requirements  for  high  vertical,  horizontal,  and  temporal  resolution  for  weather  parameters  to  support  the 
full  spectrum  of  battlefield  operations.  These  requirements  implied  a  corresponding  need  for  fine-scale. 
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high  resolution  weather  forecast  products  that  could  not  be  met  with  the  25  nmi  resolution  model  output 
that  had  been  proposed  as  a  requirement,  but  is  currently  being  provided  to  front  line  commanders.  The 
SON  stated:  “Theater  Army,  Tactical  Air  Forces,  and  Special  Operations  Forces  planners  require 
detailed  weather  observations  across  the  depth  of  the  battlefield  to  determine  mission  tactics  and  to 
manage  combat  resources.  For  example,  electro-optical  sights  and  guidance  systems,  laser  designators, 
and  night  vision  goggles  can  give  forces  a  significant  advantage.  However,  these  force -multipliers’  can 
be  significantly  limited  or  completely  neutralized  by  battlefield  induced  haze  and  smoke  and 
atmospheric  conditions  such  as  fog,  heavy  rainfall,  and  cloud  cover.  Detailed  knowledge  of  the 
existence  of  these  elements  is  essential  for  mission  planners  to  manage  these  limited  electro-optical 
assets"  (page  1,  par.  1(b)  1).  These  types  of  weather  support  requirements  could  riot  be  met  with  25  nmi 
resolution.  Paragraph  2a(l)  of  the  SON  cited  deficiencies  in  current  tactical  weather  observing 
capabilities.  It  stated  that  1)  tactical  combat  forces  require  weather  observations  of  more  weather 
elements,  with  increased  observation  frequency,  and  from  a  denser  network  of  locatioris  to  provide  an 
enhanced  current  picture  of  the  environment,  and  2)  there  is  a  corresponding  need  to  improve  forecasts. 
In  2a(l)(b),  the  SON  noted  that  AWS  could  provide  hourly  tactical  weather  observations  from  a  limited 
number  of  airfields  and  command  posts  within  a  theater  of  operations.  The  primary  deficiency  of  this 
observing  network  was  its  observation  frequency  and  the  locations  were  often  too  coarse  to  be  used  in 
detailed  tactical  applications.  This  again  implied  the  need  for  weather  products  with  more  spatial  and 
temporal  resolution  than  specified  in  the  ORD  or  the  DNA-AFGWC  model  verification  requirements 
letter  (discussed  later). 


2.5.3  Statement  of  Need  (SON)  for  Tactical  Forecast  System  (TFS)  (USAF  212-89). 

The  SON  (paragraph  2)  stated  that  AWS  must  provide  more  environmental  information  with  finer 
resolution  in  shorter  periods  of  time  to  theater  war  fighting  elements.  With  existing  or  programmed 
capabilities,  AWS  was  not  at  that  time  able  to  meet  this  requirement.  To  accomplish  the  support 
required,  AWS  needed  a  portable,  stand-alone  capability  with  greater  employment  flexibility  to  produce 
and  disseminate  weather  products.  At  the  time  this  document  was  written,  tactical  weather  support  was 
forced  to  rely  on  a  single  CONUS-based  source  —  AFGWC  — for  most  required  weather  data,  which 
left  it  vulnerable  to  a  single  point  of  failure.  The  support  structure  also  had  potential  single  points  of 
failure  in  its  communications  system,  the  Automated  Weather  Net  (AWN).  The  TFS  was  needed  to 
eliminate  complete  dependency  on  AFGWC  and  the  AWN  and  provide  a  mobile,  survivable  theater 
capability  to  rapidly  develop  and  transfer  environmental  information  to  Air-Land  Battle  components. 

Under  “current  capabilities”  in  section  3,  the  SON  stated  that  AFGWC  had  the  RWM  with  25  nmi 
resolution  for  use  in  contingency  support.  It  also  noted  that  the  Mesoscale  Weather  Prediction  and 
Analysis  System  (MPAS)  was  a  software  development  effort  that  was  needed  to  provide  theater-wide, 
short-range,  fine-scale  forecasting.  The  MPAS  software  was  required  to  run  on  transportable  or  tactical 
computers.  These  requirements  implied  that  a  25  nmi  grid  scale  was  not  sufficient  for  sustained  support 
within  theater,  and  they  support  the  stated  requirement  for  a  mesoscale  model  with  better  than  25  nmi 
resolution  running  in  theater. 
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Table  2-3.  ORD  Theater  Forecast  Model  requirements  RCM. 


System  Characteristics 

ORD  1 

Threshold 

ORD  1 
Objective 

ORD  3 

Threshold 

ORD  3 
Objective 

a.l  Source 

GFE 

a.2  Coverage 

b.  Init  GWC  Model 

yes 

yes 

c.  Init  Climo 

yes 

yes 

d.  Use  Terrain  DB 

yes 

yes 

e.  Theater  OBS 

yes 

yes 

f.  Forecast  Times 

3  hr  out  to  72  hr 

3  hr  out  to  72  hr ' 

3  hr  out  to  72  hr 

g.  Model  Run-time 

1  hour 

1  hour 

1  hour 

h.  Graceful  Degrade 

yes 

yes 

i.  Forecast  Parameters: 

Horizontal  Resolution 

1/8  Mesh(25  nmi) 

6  nmi 

Vertical  Resolution 

Variable 

Section  4  of  the  SON  listed  needed  operational  capabilities  that  are  key  to  the  selection  of  a  tactical 
forecast  model.  These  are  listed  below: 

•  The  TFS  must  be  able  to  receive  and  process  all  available  weather  data  automatically.  This  implies  a 
requirement  for  the  TFM  to  have  Four  Dimensional  Data  Assimilation  (FDDA). 

•  The  TFS  will  prepare,  quality  control,  and  analyze  data  using  an  automated  capability  that  ensures  all 
relevant  information  will  be  considered  for  weather  analyses  and  forecasts.  The  TFS  should  run 
limited  area  analysis  and  forecast  models  that  use  standard  digital  terrain  data.  The  models  will  use 
boundary  values  from  large  scale  analyses  and  forecast  fields  and  will  incorporate  theater 
observations  on  a  continuous  basis.  Based  on  available  data — either  a  complete,  partial,  or  single 
station  data  set — an  automated  capability  will  select  appropriate  models  to  produce  forecasts  and 
ensure  graceful  degradation  of  forecast  support  as  sources  of  weather  data  are  lost.  The  TFS  models 
will  be  capable  of  providing  limited  area,  high-resolution  forecasts  out  to  96  hours.  This  section  is 
the  basis  for  the  TFM  using  AFGWC  terrain  and  other  databases  for  initialization.  It  is  also  the  basis 
for  the  requirement  that  the  TFM  degrade  gracefully  when  experiencing  data-dropout  or  data-denial. 

•  For  future  requirements,  the  TFS  will  be  capable  of  adding  new  forecast  techniques  and  algorithms 
in  a  modular  fashion.  The  TFM,  as  part  of  the  TFS,  must  also  be  modular.  Table  2-4.  is  extracted 
from  the  SON  RCM  and  summarizes  TFS  requirements  that  are  related  to  or  imply  TFM  model 
selection  test  and  selection  criteria. 

2.5.4  Theater  Forecast  Model  (TFM)  Statement  Of  Work  (SOW)  (1994). 

The  following  are  excerpts  from  the  SOW  that  relate  to  or  imply  model  test  requirements. 

Ref.  SOW  Task  2,  Model  Analysis  and  Selection:  The  Contractor  shall  investigate,  analyze,  and  develop 
methods  for  evaluation  and  analysis  of  the  candidate  models  for  in-theater  weather  prediction  use  at  the 
operational  unit.  Simultaneously,  the  Contractor  shall  determine  the  computer  workstation  environment 
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that  best  meets  the  requirements  of  the  system.  This  implies  the  need  to  document  model  memory, 
storage  and  run-time  requirements  during  the  testing  and  model  comparisons. 


Table  2-4.  SON  TFS  requirements. 


Parameter 

Requirement 

Accuracy 

1.  Initialize  with  AFGWC  inputs 

Yes 

2.  Area  Coverage 

1500x1500  km 

3.  Low  Level  Temp 

SFC  to  3km 

2  km  Horizontal 

0.2  km  Vertical 

±5C  at  48  hours 

4.  Low  Level  Wind 

SFC  to  3  km 

2  km  Horizontal 

0.2  km  Vertical 

±8  kts  at  48  (r)and  96  hrs(g) 

±15  degs  at  48  (r)  and  96  hrs(g) 

5.  Low  Level  Moisture  SFC  to  3 
km 

2  km  Horizontal 

0.2  km  Vertical 

±2  gm/m3 

6.  Precip  (Rate) 

2  km  Horizontal 

±  10  mm/hr 

7.  Precip  (Total) 

2  km  Horizontal 

±  15  mm 

8.  Severe  Weather 

2  km  Horizontal 

Probability  of  detection  •  80% 

9.  Upper  Air  Winds 

2  km  Horizontal 

0.2  km  Vertical 

±  10  kts  at  48  (r)  and  96  hrs(g) 

±15  degs  at  48  (r)  and  96  hrs(g) 

10.  Pressure 

2  km  Horizontal 
for  Standard  Levels 

±  1  mb 

1 1 .  Model  Processing  Time 

All  Parameters  for  full 
theater  96  hour  forecast 

1  hour  (r) 

0.5  hour  (g) 

(r)  =  requirement  (g)  =  goal 


Ref.  SOW  Subtask  2.1,  Analyze  TFM  Model  Requirements:  The  Contractor  shall  perform  a  detailed 
requirements  review  of  the  TFM  candidate  models,  make  assessments  of  how  each  model  would 
perform  in  a  workstation  environment,  and  detail  what,  if  any,  model  performance  capabilities  would  be 
degraded  by  operating  the  model  on  a  workstation.  The  assessment  will  be  combined  with  the 
benchmark  statistics  and  presented  in  both  briefing  and  technical  report  forms.  This  implies  the  need  to 
include  model  performance  on  a  workstation  as  a  test  criterion.  Much  of  the  initial  comparison  work 
will  be  done  on  a  supercomputer  platform.  Model  run-times  will  be  carefully  tracked  on  the  CRAY  and 
used  to  estimate  the  models’  performance  on  workstations.  NCAR  did  this  type  of  analysis  for  the 
NCAR  Global  Climate  Model  (GCM)  on  various  platforms,  as  shown  in  Table  2-5.  This  type  of  • 
comparison  will  be  derived  from  the  initial  model  comparison  runs  done  on  the  CRAY. 

Ref.  SOW  Subtask  2.2,  Benchmark  Candidate  Models:  For  benchmark  testing  each  model  shall  be 
considered  to  consist  of  preprocessing  (data  analysis),  processing  (initialization  and  computation),  and 
post  processing  functions  (data  analysis  and  displays).  This  requires  test  and  comparison  of  the  three 
primary  subfunctions  of  each  model. 

This  section  of  the  SOW  also  stated  that  both  objective  and  subjective  tests  should  be  performed. 
Examples  of  such  tests  are  1)  statistical  comparisons  of  model-generated  fields  with  ground  truth 
observations  and  2)  model  predictions  of  pressure  center  locations.  Tests  would  be  performed  on  all 
required  weather  parameters  outlined  in  the  TFM  requirement  documents.  Those  parameters  include  air 
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temperature  and  dew  point  depression  within  2  degrees  Celsius,  wind  speed  within  2  knots  at  speeds  less 
than  20  knots  and  within  5  knots  at  speeds  greater  than  20  knots,  wind  direction  within  30  degrees,  and 
surface  pressure  within  1.7  millibars.  These  criteria,  while  quantitative,  are  really  aimed  at  each  model’s 
ability  to  predict  pressure  fields  and  resultant  winds.  Together,  these  parameters  provide  an  indication  of 
the  model’s  ability  to  forecast  the  development  and  location  of  overall  synoptic  and  mesoscale  features. 


Table  2-5.  (Haack,  1993).  CCM1  run  performance  vs  platform. 


Platform 

Performance 

CRAY  YMP 

126% 

CRAY-XMP 

100% 

IBM  RS  6000/550 

12% 

SUN  4/260 

1.6% 

DEC  VAX  8800 

1.4% 

Ref.  SOW  Subtask  2.3,  Computer  Technology  and  Hardware  Selection:  Although  this  section  discussed 
the  work  platform  requirements  to  host  the  TFM,  it  also  implied  a  requirement  to  consider  the  structure, 
size,  and  input/output  of  each  model  to  make  sure  it  can  be  ported  to  a  workstation  environment.  This 
section  also  levied  a  requirement  to  assess  model  performance  when  processing  regions  that  are  data- 
sparse,  data-denied,  or  data-void.  The  SOW  called  for  these  assessments  to  be  made  to  simulate  model 
operation,  assimilation  of  data,  production  of  a  forecast,  and  using  the  forecast  to  initialize  the  next 
model  mn.  The  production  cycle  for  this  test  criterion  is  5  days.  The  SOW  further  required  that  this 
simulation  cycle  allow  time  to  examine  the  effects  of  varying  data  densities. 

Ref.  SOW  Task  3,  Model  Adaptation,  Testing,  and  Validation:  The  TFM  objective  goal  for  model 
resolution  is  a  horizontal  resolution  of  10  nmi  and  a  vertical  resolution  of  30  feet  below  3000  feet 
altitude  and  1000  feet  above  that. 

Ref.  SOW  Subtask  3.2,  Test  and  Validation:  This  section  required  that  the  system/model  verification 
procedure  1 )  prove  that  the  system/model  implementation  meets  specifications  in  the  ORD  and  2)  show 
that  the  forecast  model  produces  an  accurate  representation  of  the  atmosphere.  It  must  also  ensure  that 
the  adapted  model  accurately  represents  the  original  model.  This  requirement  implied  a  subjective 
criterion  of  accurately  describing  the  atmosphere  and  a  selection  requirement  that  the  initial. 
Government  Furnished  Equipment  (GFE)-supplied  model  is  sufficiently  well  documented  to  allow 
downsizing  and  restructuring  to  ran  on  the  workstation  platform  selected  to  host  the  TFM. 


2.5.5  DSWA  Test  Requirements,  DNA  Letter  dated  21  June  1994. 

Requirements  that  were  derived  from  this  document  (DNA,  1994)  are  as  follows. 
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2.5.5. 1  DSWA  Desired  Model  Characteristics.  The  model  selected  will  be  required  to  run  both  at 
AFGWC  and  on  a  workstation.  This  implied  a  requirement  that  it  be  coded  to  allow  the  ma-rimnm 
degree  of  portability.  Thus  the  subjective  selection  criteria,  model  structure,  documentation,  and 
portability  (programming  language)  needed  to  be  considered.  This  letter  also  required  that  the  selected 
model  must  exceed  the  current  capability  of  the  AFGWC  RWM.  Table  2-6  lists  desired  characteristics  of 
the  selected  model. 


Table  2-6.  DSWA  and  AFGWC  desired  TFM  model  characteristics. 


Model  Characteristic 

Required 

Goal 

Window  Size 

1500  x  1500  nmi 

3200  x  3200  nmi 

Horizontal  Resolution 

25  nmi  * 

6  nmi 

Vertical  Layers 

20 

none  stated 

Run-time/Cycle 

1  hour  from  ingest 

none  stated 

Forecast  Cycles 

2  per  day 

8  per  day 

These  required  or  desired  characteristics  for  the  TFM  were  translated  into  subjective  model  selection 
criteria.  Each  candidate  model  was  evaluated  regarding  its  current  ability  to  meet  these  requirements 
and  the  ability  of  future  upgrades  to  meet  future  operational  goals.  These  characteristics  also  implied 
model  run-time  and  selection  criteria  that  consider  the  ability  to  increase  the  grid  size  and  number  of 
time  steps  while  still  keeping  the  model  run-time  at  one  hour  or  less. 

2. 5.5. 2  DSWA  Desired  Model  Test  Parameters.  The  DNA  letter  includes  a  brief  TFM  evaluation  plan 
developed  at  a  joint  DSWA- AFGWC  Technical  Interchange  Meeting  (TIM)  and  lists  grid,  boundary, 
initialization,  and  output  parameters  for  the  model  comparisons.  These  are  broken  down  into  the 
following  categories: 

•  Model  Configuration: 

These  requirements  cover  the  initialization  and  timing  requirements  for  the  model  comparisons, 
as  shown  in  Table  2-7. 


Table  2-7.  DSWA-AFGWC  configuration. 


Criteria 

Value/Description 

Horizontal  grid  spacing 

25  nmi 

Computational  Domain  Grid  Points 

71  x71 

Verification  Domain  Grid  Points 

61  x61  (1500  x  1500  nmi) 

Vertical  Levels 

Not  to  Exceed  20 

Nested  Grids 

None 
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•  Model  Run  Options: 

Model  run  options  listed  in  the  evaluation  plan  are: 

—  AFGWC  will  provide  observations  and  first-guess  fields  for  initial  conditions  and 
lateral  boundary  conditions  for  comparison  tests; 

—  Each  model  will  use  whichever  observation  types/variables  it  can  incorporate  into  its 
analysis  scheme; 

—  Model  options  will  be  determined  by  a  model  expert  for  each  model  and  listed  in  the 
test  plan.  A  model  expert  is  a  person  from  the  supplying  agency  who  is  familiar  with 
the  model  and  its  optimization  to  produce  its  best  results. 

•  Theater  Weather  Regimes  for  Model  Test  Runs 

Theater  and  model  test  run  criteria  are: 

—  Models  will  be  tested  in  five  theaters  that  represent  different  climatic  and  data  density 
regimes.  These  theaters  are  Alaska,  Central  America,  ,  the  Middle  East,  Korea,  and 
CONUS. 

—  Each  model  will  be  run  for  the  00Z  and  12Z  cycles  for  each  season,  with  data  saves  for 
3  days  to  cover  the  model  verification  period. 

—  AFGWC  RWM  will  be  used  as  the  baseline  for  comparison  of  each  model  in  each 
weather  regime  and  theater.  A  TFM  candidate  must  outperform  the  RWM  to  be 
considered  viable. 

•  Verification  Criteria: 

Model  verification  criteria  agreed  to  at  the  TIM  are  listed  in  Table  2-8. 

If  there  are  conflicts  among  the  various  sources  of  model  requirements  and  test  criteria,  those  of  the 

DSWA-AFGWC  TIM  were  given  preference. 


Table  2-8.  DSWA-AFGWC  model  verification  criteria. 


Criteria 

Value/Description 

Forecast  Times 

F00,  F06,  FI 2,  F24,  F36  hours 

Output  variables 

q,  u,  v,  t,  sip,  rh 

Levels 

SFC(2m)  +  mandatory  levels:  1000, 925,  850, 700,  500, 

400,  300,  200,  150,  and  100  mbs 

Measures  of  accuracy 
(Quantitative) 

RMSE  and  RMSVE  from  available  RAOBS,.Mean 
absolute  error.  Relative  error,  Bias. 

Measures  of  accuracy 
(Subjective) 

Standard  Map  sets  for  same  time  periods  to  include  the 
following  levels:  sip,  850, 700, 500,  300  mb 

Map  Evaluation 

Review  by  AWS  Personnel 

Timing  Statistics 

Recorded  for  all  model  runs 

Technology  Transition  and 
Insertion  into  each  model 

Subjective  criteria 

Review  by  AWS/DSWA 
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2.5.6  Weather  Support  for  Army  Tactical  Operations  (FM  34-81/AFM  105-4, 1989). 

This  Training  and  Doctrine  Command  (TRADOC)  field  manual  described  weather  support  concepts  and 
detailed  weather  support  requirements  for  a  full  spectrum  of  Air-Land  Battle  operations.  Several  key 
aspects  of  weather  support  related  to  the  TFM  have  been  extracted  and  listed  below.  Key  among  these 
are  those  that  imply  the  need  for  as  fine  a  mesh  as  practical  and  the  need  to  produce  longer  range 
forecasts.  The  following  requirements  are  listed  to  ensure  that  Army  requirements  are  considered  in  the 
development  of  the  TFM  test  and  selection  criteria. 

•  Area  of  Operations: 

This  manual  foresees  that  Central  Europe,  the  Middle  East,  and  Korea  are  likely  regions  in  which 
tactical  forces  may  be  employed  in  the  foreseeable  future  (page  ii).  This  provides  a  good  fit  for 
the  proposed  test  regions  selected  by  DSWA  and  AFGWC. 

•  Forecast  Length: 

Forecasts  of  up  to  72  hours  (pages  1-8)  are  needed  to  plan  and  manage  the  engagement  of  forces. 
Forecasts  of  96  hours  are  needed  at  the  Corps  level  to  assess  the  effects  of  weather  on  planning, 
tactics,  and  predicting  the  effects  of  weather  on  enemy  operations. 

•  Mesh  and  Boundary  Layer  Requirements: 

The  manual  notes  (pages  1-9)  the  requirement  for  wind  inputs  into  the  Tactical  Fire  Direction 
Computer  System  to  support  artillery  operations,  and  notes  the  need  for  wind  forecasts  to  support 
chemical  and  nuclear  dispersion  forecasting.  It  also  indicates  a  need  for  accurate  modeling  of 
planetary  boundary  layer  conditions.  This  implies  the  need  for  as  fine  a  mesh  as  is  practical- 
nesting)  within  an  Area  of  Operations. 

•  Model  Robustness  to  Data  Sparseness: 

On  page  2-1,  the  manual  discusses  the  problem  of  denial  of  weather  data  and  data  sparseness  in 
the  area  of  operations.  This  implies  a  need  for  a  TFM  weather  model  to  be  robust  and  degrade 
gracefully  when  these  conditions  are  encountered. 

•  Need  for  In-Theater  Model  Support: 

Page  3-1  discusses  disadvantages  of  centralized  support  because  of  possible  loss  of 
communications  to  forecast  centers.  It  also  discusses  past  use  of  coarse-grid  global-scale  forecast 
model  output  provided  by  forecast  centers. 

•  Chemical  Warfare  Operations: 

In  a  summary  of  staff  weather  officer  support  roles  on  pages  3-7,  the  manual  discusses  the  need 
for  weather  forecast  support  to  chemical  warfare  operations,  including  prediction  of  low  level 
inversions,  wind  shears,  wind  shifts,  and  diurnal  weather  changes  within  the  boundary  layer. 
Micro-scale  terrain  effects  on  these  types  of  operations  also  must  be  accounted  for  in  forecast 
model  support.  This  implies  the  need  for  a  fine  mesh  (nesting)  and  a  model  that  can  produce 
reasonable  forecasts  of  the  planetary  boundary  layer. 

2.5.7  CWS  Technical  Alternatives  Interim  Report,  ESC/XRC,  (1993). 

The  purposes  of  this  study  by  ESC  were  to  determine  analysis  and  forecast  model  outputs  needed  for 
tactical  weather  support,  to  review  existing  and  planned  mesoscale/microscale  meteorological  modeling 
efforts  and  their  suitability  for  use  in  tactical  forecasting,  and  to  determine  additional  software  and 
models  needed  to  meet  battlefield  requirements.  The  following  are  key  points  of  this  study  germane  to 
this  effort. 
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Operations  Concept: 

The  operations  concept  for  theater  weather  support  is  for  AFGWC  to  provide  initial  forecasts 
while  weather  support  teams  arrive  and  set  up  their  operations  in-theater.  After  a  theater  weather 
forecast  support  function  is  established,  forecast  support  will  transition  to  a  mesoscale  model  run 
at  the  Theater  Forecast  Unit  for  the  battlefield  Area  of  Operations.  The  tactical  forecast 
mesoscale  model  will  be  driven  by  surface  observations  taken  in-theater  and  any  remotely  sensed 
data,  as  might  be  available  from  platforms  such  as  Defense  Meteorological  Satellite  Program 
(DMSP). 

Source  of  Stated  Forecast  Requirements: 

Requirements  for  forecast  and  analysis  support  stated  in  the  ORD,  the  Weather  2000  Study,  and 
the  HQ  TAC  Plan  for  Automated  Support  are  based  on  needs  for  the  observation- of  each 
environmental  element,  not  as  a  specific  forecast  parameter  produced  by  a. model.  Therefore, 
model  requirements  tend  to  be  overstated  regarding  a  model’s  ability  to  produce  particular 
forecast  parameters  desired  for  operational  support. 

Model  Outputs: 

As  part  of  their  discussion  of  a  notional  forecast  model,  the  ESC  study  notes  that  all  numerical 
weather  prediction  models  typically  provide  the  same  (or  closely  related)  set  of  basic  output 
variables:  horizontal  components  of  the  wind  (u  and  v);  the  vertical  component  of  the  wind  (w); 
potential  temperature  (0);  specific  humidity  (q);  and  pressure  (p).  Other  desired  atmospheric 
elements  can  be  derived  from  this  set  of  variables.  Close  comparisons  of  these  parameters  using 
the  MRC-ASTeR  model  verification  software  will  be  one  of  the  primary  set  of  test  criteria.  The 
types  of  statistics  produced  by  the  MRC-ASTeR  REVU  verification  software  are  discussed  in 
Section  4.6. 

Model  Outputs  vs.  Model  Capabilities: 

Attachments  4  and  5  of  the  ESC  study  tabulate  present  numerical  model  capabilities  and  tactical 
forecast  model  essential  Elements.  An  excerpt  from  these  tables,  shown  in  Table  2-9,  provides  a 
brief  summary  of  known  model  capabilities  and  the  types  of  outputs  deemed  essential  for  a 
theater  forecast  model.  These  provide  insight  into  the  final  determination  of  reasonable  selection 
criteria  for  the  TFM  comparison  testing  by  showing  what  the  most  important  customer  support 
requirements  are  in  priority  order  and  by  indicating  how  each  desired  parameter  is  produced 
(either  directly  or  indirectly  by  the  forecast  model). 


Table  2-9.  Forecast  model  output  vs.  priority  and  model  capability,  [from  ESC  study,  (1993)]. 


Forecast  Product/Element 


Wx  2000 Priority 


Model 


- 

Code* 

1 

1 

Temperature  ( Runway) 

1 

3 

Temperature  (Freezing  Level) 

1 

3 

2 

1 

Thermal  Stress  (Wind  Chill  -  Heat  Stress) 

2 

3 

Low  Level  Wind  Shear 

2 

3 

Low  Level  Wind  Shear 


Table  2-9.  Forecast  model  output  vs.  priority  and  model  capability, 
[from  ESC  study,  (1993)].  (Continued). 


Forecast  Product/Element 

Wx  2000  Priority 

Model 

Code* 

Low  Level  Wind  (Drop  Zone  Mean  Wind) 

2 

3 

Low  Level  Moisture  (Specific  Humidity) 

3 

1 

Low  Level  Moisture  (Absolute  Humidity,  Dew  Point,  RH) 

3 

3 

Low  Level  Temperature  (Surface) 

3 

3 

4 

1 

Upper  Air  Temperature  (Trop  Ht,  Inversion  Ht,  Mixing  Depth,  Freezing 

Level,  Vertical  Profile,  Gradients(Vertical  &  Horizontal),  D  Values) 

4 

3 

5 

1 

Upper  Air  Wind  (Vertical  and  Horizontal  Gradients,  Wind  shear, 

Turbulence) 

5 

3 

Upper  Air  Moisture  (Specific  Humidity) 

6 

1 

Upper  Air  Moisture  (Condensation  Pressure  Spread,  Absolute  Humidity, 
Dewpoint,  RH) 

6 

3 

7 

2 

Precipitation  Rate 

8 

1 

Present  Weather  (Type  and  Intensity) 

8 

4 

Severe  Weather  (Stability  Index) 

9 

3 

Severe  Weather  (Snow  >6  ins,  moderate  snow) 

9 

4 

Severe  Weather  (Winds  Greater  than  30,  35,  40,  50  kts) 

9 

4  * 

Severe  Weather  (Freezing  Precip) 

9 

4 

9 

4 

Severe  Weather  (Thunderstorms,  Lightning,  Tornadoes,  Thunderstorm  Tops) 

9 

5 

Cloud  Cover 

10 

2 

Sea  State 

17 

4 

Wave  Height 

18 

4 

19 

4 

Low  Level  Temperature  (Inversion  Height) 

20 

2 

Low  Level  Temperature  (Vertical  Profile) 

20 

3 

Low  Level  Temperature  (Horizontal  &  Vertical  Gradients) 

20 

3 

Refractive  Index 

21  ' 

4 

Cloud  Phase 

22 

2 

Cloud  LWC 

23 

2 

State  of  Surface  (Snow  Cover-Depth,  Age,  Character) 

24 

2 

Cloud  Amount 

25 

2 

*  Model  Capability  Codes 

1  =  Forecast  element  is  directly  forecast  by  numerical  models 

2  =  Some  but  not  all  models  produce  this  parameter  directly 

3  =  Forecast  element  is  derived  from  model  output  via  some  simple  relationship 

4  =  Derived  from  model  output  via  algorithm  with  accurate  results 

5  =  Derived  from  model  output  via  algorithm,  but  results  not  reliable  or  contain  some  uncertainty 
2.6  REQUIREMENTS  STUDY. 

Based  on  the  review  of  the  documents  cited,  a  matrix  of  recommended  test  requirements  was  developed. 
This  matrix  is  shown  in  Table  2-10  and  has  the  test  and  selection  criteria  sorted  into  three  classes. 

•  Class  1  Criteria: 
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These  consist  of  model  comparison  criteria  that  can  be  compared  to  a  quantifiable  standard.  The 
order  of  consideration  is  first  to  compare  basic  model  output  parameters  (t,  p,  u,  v,  w,  q)  to 
rawinsonde  and  surface  data  using  the  REVU  model  performance  software.  This  will  provide 
insight  on  each  model’s  ability  to  correctly  forecast  basic  output  parameters  at  a  given  geographic 
location.  Next,  model  analyses  will  be  compared  to  a  standard  meteorological  package  to  assess 
how  well  the  model  predicts  the  actual  location  and  development  of  major  synoptic  and 
mesoscale  features. 

Class  2  Criteria: 

These  criteria  are  subjective  and  compare  model  functionality  and  adaptability  issues.  Model 
characteristics  such  as  robustness  to  data  sparsity,  adaptation  to  use  AFGWC  input  databases, 
ability  to  produce  analysis  fields  to  support  AFGWC  operations,  and  initialization  options  are 
areas  where  the  models  must  be  assessed  subjectively  and  compared  to  the  AFGWC  RWM. 

Class  3  Criteria: 

These  subjective  criteria  cover  areas  such  as  software  management  issues,  model  adaptation  to  a 
work  station  environment,  and  the  test  scenarios  that  have  been  selected  to  test  the  models 
against  each  other  and  the  RWM.  One  of  the  key  issues  is  to  complete  a  synoptic  description  of 
all  weather  data  sets  that  will  be  used  as  test  inputs.  This  is  required  in  order  to  fully  understand 
the  behavior  of  the  models  that  are  based  on  the  actual  weather  conditions  input. 


Table  2-10.  Recommended  model  test  and  selection  requirements  matrix. 


Type  of  Criteria 

Measurement 

Compare  to 

CLS 

Source 

OBJECTIVE  Basic 

t,  p,  u,  v,  w,  q 

RAWINS 

Model  Outputs 

RMSE,  RMSVE,  mean 
absolute  Errors 

Metsat  soundingsfdata  sparse) 
REVU  comparison  software 

1 

OBJECTIVE 

Derived  Analysis 
Fields 

Cloud  Cover 

Winds 

Pressure 

Temperature 

Observed  analysis  charts 
Nephanalysis/Metsat 

Constant  level  charts 
sfc,  850,700, 500, 300  mbs 

1 

DNA 

OBJECTIVE 

Forecast  Times 

Forecast  to  72  hours 

Observations  (sfc  and  U/A) 
Compare  against  RWM 

1 

DNAORD 

FM  34-81 
TFS-SON 

SUBJECTIVE 

Synoptic  and  Mesoscale 

Surface  charts 

1 

DNA 

Atmospheric  State 

feature  location  and 
description(timing) 

Metsat 

Surface  and  U/A  Data 

SUBJECTIVE 

•Science/Physics 

Cross  compare  each  model  and 

2 

DNA 

Model  Traits  and 
Subfunctions 

•Robustness  to  Lack  of 

Inputs 

compare  to  RWM  as  the 
baseline 

2 

ORD 

SUBJECTIVE 

Initialization 

Area  Coverage 

Boundary  Condition  Options 
FDDA  Available 

Subjective  review  of  each 
model  and  cross  compare  to 
RWM 

2 

DNA 

OBJECTIVE  Basic 

t,  p,  u,  v,  w,  q 

RAWINS 

ESSHi 

Model  Outputs 

RMSE,  RMSVE,  mean 
absolute  Errors 

Metsat  soundings(data  sparse) 
REVU  comparison  software 

1 

H 

Table  2-10.  Recommended  model  test  and  selection  requirements  matrix.  (Continued). 


Type  of  Criteria 

Measurement 

Compare  to 

CLS 

Source 

OBJECTIVE 

Cloud  Cover 

Observed  analysis  charts 

1 

DNA 

Derived  Analysis 

Winds 

Nephanalysis/Metsat 

Fields 

Pressure 

Constant  level  charts 

Temperature 

sfc,  850,700, 500,  300  mbs 

OBJECTIVE 

Forecast  to  72  hours 

Observations  (sfc  and  U/A) 

1 

DNAORD 

Forecast  Times 

Compare  against  RWM 

FM  34-81 

TFS-SON 

SUBJECTIVE 

Surface  charts 

1 

DNA 

Atmospheric  State 

feature  location  and 

Metsat 

description(timing) 

Surface  and  U/A  Data 

SUBJECTIVE 

•Science/Physics 

Cross  compare  each  model  and 

2 

DNA 

Model  Traits  and 

•Robustness  to  Lack  of 

compare  to  RWM  as  the 

2 

ORD 

Subfunctions 

Inputs 

baseline 

SUBJECTIVE 

Area  Coverage 

Subjective  review  of  each 

2 

DNA 

Initialization 

Boundary  Condition  Options 

model  and  cross  compare  to 

FDDA  Available 

RWM 

SUBJECTIVE 

Documentation 

Subjective  review  of  each 

3 

Software 

Portability  &  upgradability 

model  and  cross  compare  to 

3 

Characteristics 

Analysis  package 

RWM 

3 

I/O  and  interface  to  AFGWC 

3 

Models  and  C4I 

3 

SUBJECTIVE 

Memory  Requirements 

Cross  compare  models  and 

3 

ORD 

Implementation 

Run-time 

compare  to  RWM  with  same 

Factors 

I/O 

vertical  levels  and  grid  spacing 

SUBJECTIVE 

AFGWC  DATA  BASES 

•Determine  how  well  model 

2 

DNA 

Model  Input 

•GSM  to  initialize 

interfaces  to  use  of  AFGWC 

2 

Requirements 

•GWC  sfc  and  UA  and 

databases. 

terrain  databases 

•Impacts  of  AFGWC  input 

•Metsat  (DMSP  ingest) 

databases  (scale,  etc.)  on 

model  performance 

SUBJECTIVE 

Required  Test  Scenarios 

•determine  and  document 

3 

DNA 

Test  Data  Weather 

Weather  Regimes  are: 

effects  of  test  scenarios  on 

3 

Scenarios  Impacts 

•  Alaska 

overall  model  performance. 

on  Model 

•  Central  America 

•provide  a  complete  synoptic 

Performance  and 

•  Middle  East 

analysis  of  the  overall  weather 

Model  Comparisons. 

•  Korea 

regime  for  each  test 

•CONUS 

•Compare  to  RWM 
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SECTION  3 

TEST  ITEM  DESCRIPTIONS  AND  RUN  PERFORMANCE 

This  section  describes  the  models  tested,  their  configuration  during  the  tests,  and  their  run  performance 
on  the  CRAY  and  workstation. 

3.1  SOFTWARE  MODELS  AND  CONFIGURATION. 

Each  model  was  configured  using  the  “best”  selection  of  physical  parameters  available,  our  a  priori 
assumption  being  that  this  would  produce  the  best  forecast.  Candidate  models  can  be  grouped  by  their 
use.  NORAPS6  and  RWM  were  used  in  operational  settings  and  have  few  switches  to  select  different 
physical  models.  Their  options  are  more  related  to  types  of  data  output  available.  MM5  and  RAMS 
have  more  of  a  research  background  and  have  numerous  physical  model  switches.  In  addition,  they  have 
many  of  the  same  types  of  options  for  data  assimilation  and  output.  The  four  candidate  models  have 
existed  for  some  time,  or  are  new  upgrades  of  existing  models.  RAMS  and  its  immediate  predecessors 
have  a  history  dating  back  to  the  mid-1970s.  MM5  and  NORAPS6  have  similar  lineages  that  provide  a 
track  record  of  what  they  can  and  cannot  do  and  how  well  they  can  do  it. 

RAMS,  MM5,  and  NORAPS6  have  much  in  common:  they  are  all  3-D,  primitive  equation,  relocatable, 
regional  mesoscale  models;  they  can  all  use  staggered  gridding,  terrain-following  vertical  coordinates, 
and  four  dimensional  data  assimilation  using  nudging;  and  they  include  many  similar  parameterizations 
for  sub-grid  mixing,  cumulus,  and  radiation.  There  are  some  differences,  particularly  between 
NORAPS6  and  the  other  two  models. 

The  NORAPS  series  of  codes  has  been  developed  by  the  Navy  for  their  own  operational  use  and  are 
optimized  for  ocean  area  forecasts;  MM5  and  RAMS  are  not.  This  distinction  is  easily  seen  by  noting 
that  NORAPS6  models  sea  ice,  but  RAMS  and  MM5  do  not.  RAMS  and  MM5  both  model  numerous 
types  of  land  use,  which  can  significantly  affect  local  weather,  but  NORAPS6  has  no  land  use  model. 

Two  of  the  models  that  were  tested  (NORAPS6  and  RWM)  do  not  have  options  for  a  non-hydrostatic 
equation  set.  A  non-hydrostatic  model  is  critical  in  the  simulation  of  a  variety  of  buoyancy-driven 
atmospheric  features.  For  the  grid  scales  used  in  this  study  (46  km),  neither  buoyancy-driven  or  the 
oscillatory  circulations  would  be  resolved.  A  large  thunderstorm  may  only  be  10-20  km  in  diameter,  and 
since  an  NWP  model  needs  to  have  a  circulation  defined  by  at  least  four  grid  spaces  to  be  handled 
adequately,  even  runs  with  a  grid  spacing  of  about  10  km  will  not  be  sufficiently  fine  to  resolve  these 
circulations.  A  lee  wave  train  (an  example  of  an  oscillatory  circulation)  usually  has  a  horizontal 
wavelength  of  about  20  km,  so  these  will  not  be  resolved  well  either.  At  the  resolution  used,  the 
potential  advantages  of  a  non-hydrostatic  model  will  be  minimized. 

However,  as  computers  get  faster,  the  spacing  of  the  objective  domain  will  be  able  to  have  a  finer  grid 
spacing.  Once  the  grid  spacing  is  reduced  below  the  6  nmi,  a  non-hydrostatic  model  will  become 
necessary. 

3.1.1  Grid  Configuration. 

The  objective  of  these  tests  was  to  compare  predicted  forecasts  produced  by  the  various  models.  To 
accomplish  these  comparisons  the  computational  domains  were  made  to  be  as  similar  as  possible.  There 
is  no  common  map  projection  available  for  all  the  models,  so  we  chose  projections  that  are  appropriate 
for  each  theater  and  as  similar  as  possible.  RAMS  uses  a  rotated  polar-stereographic  projection  which 
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“rotates”  the  pole  point  of  the  projection  to  the  center  of  the  computational  domain.  This  allows  the  use 
of  the  same  projection  with  a  minimum  of  distortion.  MM5,  NORAPS6,  and  RWM  do  not  use  a  rotated 
projection;  therefore,  two  projections  had  to  be  used  in  an  attempt  to  minimize  distortion.  The  three 
models  used  a  Mercator  projection  for  the  Central  America  domain  and  a  Lambert-Conformal  projection 
for  the  remaining  areas.  The  Mercator  projections  differed  slightly,  as  MM5  and  NORAPS6  are 
hardwired  to  use  the  equator  as  the  standard  latitude,  and  RWM  uses  22.5  °N. 

The  same  computational  domain  or  grid  was  used  in  all  the  tests  except  the  objective  resolutions  tests. 

A  71  x  71  grid  with  nominal  46.3  km  spacing  was  used.  The  gridded  regions  are  displayed  below 
(Figures  3-1  through  3-5).  The  two  gridded  regions  for  the  objective  tests  with  the  nested  finer 
resolution  grids  are  also  displayed  (Figures  3-6  and  3-7).  The  finer  grid  spacing  is  approximately  the 
6  nmi  objective.  RAMS  allows  for  a  user-selectable  spacing  ratio  between  the  grids,  so  it  would  be 
possible  to  take  the  coarse  grid  as  the  25  nmi  grid  used  in  the  threshold  tests  and  use  a  4-1  grid  space 
ratio.  However,  both  MM5  and  NORAPS6  allow  only  a  3-1  grid  space  ratio.  Therefore,  we 
reconfigured  the  coarse  grid  domain  with  an  18  nmi  spacing  and  ran  all  models  with  a  nested  grid  at  a  3- 
1  grid  space  ratio.  Because  of  the  finer  spacing,  we  used  98  x  98  grid  points  on  the  coarse  grid  to  cover 
the  1500  x  1500  nmi  domain.  Nested  grids  will  be  run  with  71x71  grid  points. 

Vertical  structures  also  differ.  RAMS  uses  a  terrain-following  height  coordinate  and  the  others  use  a 
terrain-following  pressure  coordinate.  However,  even  with  these  differences,  we  closely  matched  the 
“effective”  resolutions  of  the  models.  Approximately  twenty  layers  were  used  for  each  model.  Some  of 
the  models  may  have  been  designed  (or  work  better)  with  more  levels  than  the  target  of  20.  NORAPS6 
is  run  operationally  with  36.  The  target  number  of  levels,  while  adequate  in  some  cases,  should  be . 
greater  to  allow  for  increased  resolution  near  the  ground  and  additional  levels  in  the  stratosphere. 


Figure  3-1.  CONUS  grid  centered  at  38  N  and  98  W. 
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Figure  3-4.  Middle  Eastern  grid  centered  at  32.5  N  and  41.5  E. 


Figure  3-6.  Middle  Eastern  grid  with  nested  objective  grid. 


Figure  3-7.  Central  American  grid  with  nested  objective  grid. 


3.1.2  Configuration  of  “Physics”  Parameters. 


The  candidate  forecast  models  could  be  operated  in  a  number  of  different  configurations — hydrostatic, 
non-hydrostatic,  sponge  boundary,  transmitive  boundary,  etc.  The  “best”  and  most  appropriate  options 
were  selected.  Since  these  tests  were  more  concerned  with  capability  than  with  speed,  we  used  the  most 
sophisticated  level  of  parameterization  available  in  each  of  the  models  which  is  consistent  with  the  tests, 
tactical  applications  and  the  developer’s  recommendations  (the  model  configurations  were  reviewed  by 
R.  Hodur  (NORAPS6),  S.  Chen  (MM5),  and  Captain  Crasner  (RWM)).  Although  four-dimensional  data 
assimilation  was  an  available  option  for  RAMS  and  MM5,  it  was  not  used  in  the  benchmark  comparison 
tests,  because  the  AF  baseline,  RWM,  does  not  include  four-dimensional  data  assimilation.  Each  model 
will  be  described  briefly  below. 


3.2  FIFTH-GENERATION  PSU/NCAR  MESOSCALE  MODEL  (MM5). 

MM5  is  a  regional-scale  primitive  equation  model  that  can  be  configured  hydrostatically  or  non- 
hydrostatically.  It  uses  a  terrain-following  coordinate  in  pressure  and  solves  its  finite-difference 
equations  with  a  time-split  scheme  using  the  leapfrog  operator.  Multiple,  moving,  overlapping  nesting 
capability  exists  with  two-way  interactivity  and  predefined  nest  ratios  of  3: 1.  Its  boundary-layer  physics 
package  can  be  either  a  simple  bulk  aerodynamic  parameterization  or  a  more  detailed  scheme  based 
upon  a  revised  version  of  Blackadar's  PBL  model  (Zhang  and  Anthes,  1982).  The  atmospheric  radiation 
option  provides  longwave  and  shortwave  schemes  that  interact  with  the  atmosphere,  including  cloud  and 
precipitation  fields,  as  well  as  with  the  surface  (Dudhia,  1989). 

Large-scale  and  convective  precipitation  modules  are  included  in  the  model,  and  large-scale  processes 
are  treated  explicidy.  Marshall-Palmer  size  distributions  are  assumed  for  rain  and  snow  and  solid  and 
liquid  water  are  allowed  to  co-exist.  Options  for  deep  cumulus  convection  include  parameterizations 
based  on  Kuo  (1974),  and  a  modified  Arakawa-Schubert  (1974)  scheme  that  includes  moist  convective- 
scale  down  draft  (Grell,  1991). 


3.2.1  MM5  Configuration. 


Grid  Projection  and  Vertical  Levels 


MM5  has  polar-stereographic,  Lambert  conformal,  and  Mercator  projection  options.  We  used  the 
Lambert  conformal  projection  for  the  CONUS,  Korea,  Middle  East,  and  Alaska  domains  and  the 
Mercator  projection  for  the  Central  America  region.  MM5  permits  any  number  of  vertical  levels  to  be 


used.  We  have  used  23  layers  for  these  tests.  The  vertical  levels  selected  are  cp:  1.00, 0.99, 0.98, 0.96, 
0.93, 0.89, 0.85, 0.8, 0.75, 0.7, 0.65, 0.6, 0.55, 0.5, 0.45, 0.4, 0.35, 0.3, 0.25,  0.2, 0.15, 0.1, 0.05,  and  0.0. 


Computational  or  “ Physics  ”  Options 

Table  3- 1 .  gives  the  physics  and  numeric  features  that  are  available  as  options  in  the  standard  version 
and  those  that  we  chose  to  activate  for  the  tests. 
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Table  3-1.  MM5  options. 
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Table  3-1.  MM5  options  (Continued). 


MODEL  PARAMETER 

OPTIONS 

USED 

TOP  BOUNDARY 
CONDITIONS 

Rigid 

✓ 

Free  surface 

Sponge 

SURFACE 

PARAMETERIZATIONS 

Bulk  PBL 

High-resolution  Blackadar  PBL 

BATS  vegetation  parameterization 

s 

SUBGRID  DIFFUSION 
PARAMETERIZATIONS 

Vertical  mixing  in  conjunction  with  PBL 
parameterization  only 

s 

DEEP  CONVECTION 
PARAMETERIZATIONS 

Anthes-Kuo 

Arakawa-Schubert 

Fritsch-Chappell 

Kane-Fritsch 

Grell 

V 

RADIATION 

PARAMETERIZATIONS 

Long  and  shortwave  scheme  with  cloud  effects 

V 

Longwave  scheme  with  atmospheric  tendencies 

s 

RESOLVED 

CONDENSATION 

PARAMETERIZATIONS 

Bulk  microphysics  parameterization  with  liquid  and 
2  ice  species 

V 

3.2.2  MM5  Model  Source. 

Two  versions  of  MM5  were  obtained:  the  CRAY  source  code  from  National  Center  for  Atmospheric 
Research  (info.ucar.edu)  (downloaded  August,  1994);  and  the  workstation  source  code  from  the  National 
Center  for  Atmospheric  Research  (info.ucar.edu)  (downloaded  September,  1994). 

3.2.3  Execution  Speed. 


Because  the  MM5  observational  data  “pre-processors”  had  not  been  ported  to  a  workstation  platform 
when  the  study  was  conducted,  all  MM5  runs  were  made  on  the  CRAY  Y-MP’s  at  Los  Alamos.  The 
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execution  speed  on  the  “gamma”  machine  averaged  about  12,600  seconds,  with  the  pre-processors 
taking  another  1400  seconds.  Assuming  that  the  ratio  between  the  CRAY  and  workstation  performance 
on  NORAPS6  (ratio  =  5.17)  was  the  same  for  MM5,  the  model  run  would  have  taken  about  65,000 
seconds.  The  possibility  exists  that  although  MM5  was  developed  exclusively  for  a  CRAY,  that 
NORAPS6  has  been  better  optimized  for  a  vector  machine.  Therefore,  the  workstation  performance  of 
MM5  may  be  better  than  the  65,000-second  estimate. 

3.2.4  Attributes  Noted. 

MM5  has  a  long  history  dating  back  to  the  mid-1970s  at  PSU  and  continuing  through  the  present  with 
development  at  NCAR,  PSU,  and  other  places.  Some  of  the  code  have  not  been  modified  for  a  long  time 
and  was  written  in  the  FORTRAN  66  style  of  programming.  This  is  especially  true  of  the  “pre¬ 
processors”.  No  code  existed  for  these  data  ingestion  routines  that  allowed  them  to  work  on  a 
workstation.  At  the  time  of  the  comparison  study,  NCAR  indicated  they  were  developing  such  code  and 
a  full  workstation  capable  version  should  now  exist.  Our  attempts  to  get  the  same  routines  to  initialize 
the  multiple  grids  for  the  objective  tests  were  also  unsuccessful.  The  pre-processors  distributed  with 
MM5  that  accomplish  the  data  ingestion  were,  at  the  time  of  the  study,  incapable  of  supporting  the  TFM. 

From  a  capability  standpoint,  MM5  is  a  fairly  complete  model  and  has  been  run  extensively  over  the 
years  at  the  threshold  resolutions. 

•  Weak  points  include  : 

1.  simplistic  parameterization  of  the  boundary  layer  does  not  allow  elevated  mixed  layers,  - 

2.  simplistic  microphysics  parameterization, 

3.  the  non-hydrostatic  options  are  very  new  and  have  not  been  tested  thoroughly, 

4.  data  analysis  uses  a  very  simple  Cressman  technique  on  pressure  surfaces. 

3.2.5  Model  Support  and  Maintenance. 

MM5  is  copyrighted  by  the  University  Corporation  for  Atmospheric  Research  (UCAR)  and  is  distributed 
by  NCAR  without  license  fees.  A  minimal  level  of  support  is  given  by  NCAR  if  their  time  allows.  To 
our  knowledge,  commercial  support  is  not  available. 

3.3  NAVY  OPERATIONAL  REGIONAL  ATMOSPHERIC  PREDICTION  SYSTEM  VERSION  6 
(NORAPS6). 

NORAPS6  is  a  regional-scale,  primitive  equation,  hydrostatic  model  that  uses  a  split,  explicit  time 
integration  scheme  to  predict  dynamic  and  thermodynamic  variables.  Its  grids  use  a  terrain-following 
vertical  pressure  coordinate  system.  Leapfrog  time  and  space  differencing  is  used  with  a  fourth-order 
advection  scheme.  Surface  fluxes  are  modeled  according  to  Deardorff  (1972).  The  planetary  boundary 
layer  (PBL)  is  assumed  to  be  well-mixed,  so  all  model  levels  within  the  PBL  are  replaced  by  average 
values  through  the  PBL  each  time  the  model  physics  routines  are  called;  presently,  the  model  physics 
routines  are  called  every  nine  dynamic  time  steps. 

Large-scale  and  convective  precipitation  modules  are  included  in  the  model,  and  evaporation  of  falling 
precipitation  is  allowed  to  occur.  Its  deep  cumulus  convection  scheme  is  based  on  the  Kuo  scheme, 
which  allows  deep  moist  convection  to  occur  when  the  moisture  convergence  exceeds  some  arbitrarily 
specified  threshold.  Cloud  temperatures  and  mixing  ratios  follow  a  moist  adiabat  and,  so,  represent  a 
non-entraining  cloud  (Hodur,  1987). 
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3.3.1  N0RAPS6  Configuration. 


Projection  and  Vertical  Levels 

NORAPS6  uses  polar-stereographic,  Lambert  conformal,  Mercator,  spherical,  and  Cartesian  projection 
options.  We  used  the  Lambert  conformal  projection  for  the  CONUS,  Korea,  Middle  East,  and  Alaska 
domains  and  the  Mercator  projection  for  the  Central  America  region.  The  model  currently  uses  36 
vertical  levels  in  its  operational  calculations,  although  the  Navy  previously  used  21.  These  21  levels 
guided  the  selection  of  the  levels  used  for  these  tests  (Hodur,  personal  communication).  The  vertical 
levels  selected  are  (<yp):  1.00, 0.98,  0.95, 0.915,  0.875, 0.825, 0.775, 0.725, 0.675, 0.625, 0.55, 0.44, 
0.33, 0.22, 0.145, 0.095, 0.065, 0.035, 0.015, 0.005,  and  0.000. 

Computational  or  “Physics”  Options 

Table  3-2.  gives  the  physics  and  numeric  features  available  as  options  in  the  standard  version  and  those 
that  we  chose  to  activate  for  the  tests.  Because  NORAPS6  was  developed  as  an  operational  model,  there 
are  very  few  options  in  the  input  data  controlling  the  physics. 

Relationship  between  CRAY  and  Workstation  Version 

No  significant  changes  were  required  to  the  code  as  received.  All  components  of  the  model  compiled 
and  ran  on  the  workstation  after  acquisition  of  a  FORTRAN  90  compiler.  Minor  changes  in  the  data 
ingestion  program  were  required  to  ingest  data  in  our  format  and  to  remove  Navy-unique  data  queries. 

Table  3-2.  NORAPS6  options. 


MODEL  PARAMETER 

OPTIONS 

USED 

BASIC  EQUATIONS 

Hydrostatic 

✓ 

DIMENSIONALITY 

3-Dimensional 

✓ 

GRID 

Spacing: 

Stagger-Arakawa  C-grid  horizontal  and  vertical 

✓ 

Nesting- 1  nest,  3:1  space  ratio 

Movable  nested  grid 

Horizontal  spacings  used-variable-20km  and  greater 

V 

INITIALIZATION 

Multivariate  optimum  interpolation 

✓ 

NUMERICS 

Time  differencing: 

Advection: 

Leapfrog 

V 

V 

* 

COORDINATE  SYSTEM 
Horizontal: 

Vertical: 

Mercator  projection 

Lambert-conformal 

Polar  stereographic 

V 

Spherical 

Terrain  following  pressure 

s 

LATERAL  BOUNDARY 
CONDITIONS 

Fixed 

-Sponge _ 

V 
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Table  3-2.  NORAPS6  options.  (Continued) 


MODEL  PARAMETER 

OPTIONS 

USED 

TOP  BOUNDARY 
CONDITIONS 

Rigid 

V 

SURFACE 

PARAMETERIZATION  S 

Bulk  PBL  parameterization 

V 

Shallow  cumulus  parameterization 

✓ 

SUBGRID  DIFFUSION 
PARAMETERIZATION  S 

First  or  1 .5  order  closure  in  PBL 

V 

DEEP  CONVECTION 
PARAMETERIZATIONS 

Modified  Kuo 

V 

RADIATION 

PARAMETERIZATIONS 

Long  and  shortwave  scheme  with  cloud  effects  and 
with  atmospheric  tendencies 

V 

RESOLVED 

CONDENSATION 

PARAMETERIZATIONS 

Precipitation  if  relative  humidity  exceeds  a  threshold 

V 

3.3.2.  NORAPS6  Model  Source. 

Source  code  was  received  from  Richard  Hodur,  Naval  Research  Laboratory,  Monterey  on  27  December 
1994. 

3.3.3  Execution  Speed. 

NORAPS6  was  primarily  run  on  the  “gamma”  CRAY  Y-MP  at  Los  Alamos  National  Laboratories 
(LANL).  Comparative  runs  were  made  on  the  IBM  RS/6000  workstation  for  the  SESAME  test  cases.  It 
took  2900  seconds  for  the  data  pre-preprocessing  and  15,550  seconds  for  the  actual  forecast  model  run 
on  the  workstation  and  1 174  and  3007  seconds  on  the  CRAY. 

3.3.4  Attributes  Noted. 

The  final  version  of  NORAPS6  that  we  received  was  well  written  and  had  very  good  internal 
documentation,  although  the  external  documentation  was  dated.  Because  of  the  liberal  use  of  CRAY 
FORTRAN  extensions,  NORAPS6  required  the  use  of  either  a  CRAY  or  a  FORTRAN  90  compiler. 

The  port  to  a  UNIX  workstation  was  easily  accomplished. 

•  Weak  points  include  : 

1 .  simplistic  parameterization  of  the  boundary  layer,  does  not  allow  for  elevated  mixed  layers, 

2.  no  microphysics  parameterization;  water  above  saturation  falls  to  next  level  and  evaporates, 

3.  no  non-hydrostatic  option,  and 

4.  data  analysis  does  not  use  moisture  observations,  only  interpolation  from  large  scale  field. 
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3.3.5  Model  Support  and  Maintenance. 

N0RAPS6  was  developed  by  the  Navy  and  is  being  replaced  by  the  Coupled  Ocean  Atmosphere 
Mesoscale  Prediction  System  (COAMPS)  model  development.  Commercial  support  is  unavailable. 

3.4  REGIONAL  ATMOSPHERIC  MODELING  SYSTEM  (RAMS). 

RAMS  is  a  regional-scale  primitive  equation  model  that  can  be  configured  hydrostatically  or  non- 
hydrostatically.  The  model  uses  a  terrain-following  coordinate  in  height,  and  its  finite-difference 
equations  are  solved  with  a  time-split  scheme  using  an  optional  hybrid  time-differencing  operator.  This 
allows  desirable  characteristics  of  the  leapfrog  operator  to  be  used  on  velocity  and  pressure  variables, 
while  reserving  the  “forward-in-time”  operator  for  all  other  variables.  Multiple,  moving,  two-way  grid 
nesting  capability  exists  with  two-way  interactivity. 

RAMS’s  boundary-layer  physics  package  is  based  on  similarity  theory.  Its  atmospheric  radiation  option 
provides  longwave  and  shortwave  schemes  that  interact  with  the  atmosphere  including  cloud  and 
precipitation  fields,  as  well  as  with  the  surface. 

Large-scale  and  convective  precipitation  modules  are  included  in  the  model,  and  large-scale  processes 
are  treated  explicitly.  A  range  of  options  exists  when  specifying  how  the  microphysics  are  modeled,  and 
mixed-phase  microphysics  is  allowed.  There  are  also  two  options  for  the  deep  cumulus  convection 
scheme. 

3.4.1  RAMS  Configuration. 

Projection  and  Vertical  Levels 

RAMS  has  rotated  polar-stereographic  and  Cartesian  options.  We  used  the  rotated  polar  stereographic 
projection  which  rotates  the  “pole”  point  of  the  polar-stereographic  projection  to  any  point  on  the  globe, 
typically  the  center  of  the  model  domain,  to  minimize  distortion  anywhere  on  the  Earth  without  having 
to  deal  with  many  different  projections.  The  required  vertical  zoning  was  achieved.  The  vertical  levels 
selected  are  shown  in  Table  3-3. 


Table  3-3.  RAMS  vertical  levels. 


z(m) 

P  (Pa) 

46.7 

100586.7 

160.7 

99260.7 

309.0 

97561.8 

501.7 

95393.4 

752.2 

92641.7 

1077.8 

89175.5 

1501.1 

84849.2 

2051.5 

79509.9 

2766.9 

73018.0  | 

3706.2 

65185.9 

4852.7 

56561.3 

6061.9 

48468.1 
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Table  3-3.  RAMS  vertical  levels. 


7261.9 

41359.2 

8461.9 

35088.5 

9661.9 

29591.3 

10861.9 

24812.6 

12061.9 

20690.3 

13261.9 

17159.4 

14461.9 

14167.1 

15661.9 

11653.5 

16861.9 

9554.6 

Computational  or  “Physics  ”  Options 

Table  3-4  gives  the  physics  and  numeric  features  available  as  options  in  the  standard  version  and  those 
that  we  chose  to  activate  for  the  tests. 

Table  3-4.  RAMS  options. 


MODEL  PARAMETER 

OPTIONS 

USED 

BASIC  EQUATIONS 

Hydrostatic  time-split 

Non-Hydrostatic  time-split 

✓ 

DIMENSIONALITY 

3-Dimensional 

✓ 

2-Dimensional 

1 -Dimensional 

GRID 

Stagger-Arakawa  C-grid  horizontal  and  vertical 

✓ 

Nesting-any  number,  any  integer  space  and  timestep  ratio 

Movable  nested  grids 

Spacing: 

Horizontal  spacings  used-variable-2cm  to  300km 

✓ 

INITIALIZATION 

Hybrid  isentropic/terrain-following  objective  analysis 
package 

✓ 

NUMERICS 

Time  differencing: 

Forward-backward 

Leapfrog 

Hybrid  (combination  of  leapfrog  and  forward- 
backward 

✓ 

Advection: 

✓ 

2nd  order  forward 

✓ 

6th  order  forward 

_ 

_ 
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Table  3-4.  RAMS  options  (Continued). 


MODEL  PARAMETER 


NUMERICS 

Time  differencing: 


Advection: 


OPTIONS 


COORDINATE  SYSTEM 
Horizontal: 

Vertical: 


LATERAL  BOUNDARY 
CONDITIONS 

Radiative  conditions 


Nudging  conditions: 


TOP  BOUNDARY 
CONDITIONS 


SURFACE 

PARAMETERIZATION  S 


SUBGRID  DIFFUSION 
PARAMETERIZATION  S 


DEEP  CONVECTION 
PARAMETERIZATION  S 


RADIATION 
PARAMETERIZATION  S 


Hybrid  (combination  of  leapfrog  and  forward- 
backward 


hmmbm 


4th  order  leapfro 


2nd  order  forward 


6th  order  forward 


Cartesian 


Rotated  polar  stereographic 


Cartesian 


Terrain-following  height 


Orlanski 


Klemp-Lill 


Klemp-Wilhelmson 


Davies 


Klemp-Durran  gravity  wave  radiative 


Damping  layer  (Rayleigh  friction) 


Nudging  layer 


Similarity  theory  surface  layer 


Multi-layer  (usually  5-12)  soil  temperature  /  moisture 


Vegetation 


Mellor-Yamada  turbulent  kinetic  energy  closure 


Deardorff  turbulent  kinetic  energy  closure 


Deformation/stability-based  closure 


Modified  Kuo 


Modified  Fritsch-Cha 


Weissbluth 


Long  and  shortwave  scheme  with  cloud  effects  and 
with  atmospheric  tendencies 


Long  and  shortwave  scheme  without  cloud  effects 
and  with  atmospheric  tendencies 


Table  3-4.  RAMS  options  (Continued). 


MODEL  PARAMETER 

OPTIONS 

USED 

RESOLVED 

CONDENSATION 

PARAMETERIZATIONS 

Bulk  microphysics  parameterization  with  liquid  and 
up  to  5  ice  species 

Bin-resolving  microphysics  with  32  size  bins  for 
liquid 

Relationship  between  CRAY  and  Workstation  Version 

RAMS  is  highly  portable.  The  same  version  was  tested  and  used  on  both  platforms,  although  most  runs 
were  performed  on  the  workstation. 

3.4.2  RAMS  Model  Source. 

RAMS  Version  3B  was  obtained  from  the  ASTeR  Division  of  MRC,  in  September  1994. 

3.4.3  Execution  Speed. 

Because  of  the  level  of  physics  used,  RAMS  averaged  about  31,500  CPU  seconds  on  the  IBM  RS/6000 
Model  370  workstation  for  a  36  hour  forecast  run  (about  24.3%  of  real  time).  The  data  analysis  and 
initialization  took  an  additional  35  minutes. 

3.4.4  Attributes  Noted. 

RAMS  development  dates  back  to  the  early  1970s,  but  the  vast  majority  of  the  code  has  been  kept  up  to 
date,  with  a  re-write  started  in  1986. 

•  Weak  points  include  : 

1 .  longer  execution  times  than  RWM  and  NORAPS6  due  to  more  complex  physical 
parameterizations, 

2.  simple  Barnes  analysis  used  for  data  analysis,  although  performed  on  isentropic  and  terrain- 
following  coordinates. 

3.4.5  Model  Support  and  Maintenance. 

RAMS  is  copyrighted  by  CSU,  who  have  licensed  MRC  as  the  exclusive  distributor.  The  model  is 
supported  and  maintained  by  MRC. 

3.5  RELOCATABLE  WINDOW  MODEL  (RWM). 

The  RWM  is  a  regional-scale  primitive  equation  model  that  uses  a  quasi-Lagrangian  advection  scheme 
to  predict  u  and  v  wind  components,  potential  temperature,  surface  pressure  and  specific  humidity.  It 
employs  a  single,  unstaggered  horizontal  grid  and  uses  a  terrain-following  vertical  coordinate  system. 

RWM  uses  a  basic  boundary-layer  physics  package  in  which  bulk  transfers  of  sensible  and  latent  heat  are 
allowed  over  the  ocean  when  the  sea  surface  temperature  is  greater  than  the  air  temperature  and  surface 
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friction  is  modeled  with  a  terrain-dependent  drag  coefficient.  Large-scale  and  convective  precipitation 
modules  are  included  in  the  model,  and  evaporation  of  falling  precipitation  is  allowed  to  occur.  Its  deep 
cumulus  convection  scheme  is  based  on  the  Kuo  scheme,  which  allows  deep  moist  convection  to  occur 
when  a  saturated  parcel  originating  in  the  lower  atmosphere  can  rise  through  at  least  two  contiguous 
vertical  layers  above  the  Lifting  Condensation  Level  (LCL).  RWM  did  not  account  for  land  surface 
processes,  diagnosed  cloud  fraction  and  solar/terrestrial  radiative  processes  until  recently  with  the 
inclusion  of  the  Swedish  physics  package. 

3.5.1  RWM  Configuration. 


Projection  and  Vertical  Zoning 

RWM  has  polar-stereographic,  Lambert  conformal,  and  Mercator  projections  available.  We  used  the 
Lambert  conformal  projection  for  the  CONUS,  Korea,  Middle  East,  and  Alaska  domains  and  the 
Mercator  projection  for  the  Central  America  region.  The  vertical  levels  selected  are  (CJp):  1.00, 0.98, 
0.95, 0.915, 0.875, 0.825, 0.775, 0.725, 0.675, 0.625, 0.55, 0.44, 0.33, 0.22, 0.145, 0.095, 0.065, 0.035, 
0.015,0.005,  and  0.000. 

Computational  or  “Physics”  Options 

Table  3-5.  gives  the  physics  and  numeric  features  available  in  RWM.  The  model  is  an  operational 
model  with  no  user  options. 


Table  3-5.  RWM  options. 


MODEL  PARAMETER 

OPTIONS 

USED 

BASIC  EQUATIONS 

Hydrostatic 

V 

DIMENSIONALITY 

3-Dimensional 

s 

GRID 

Stagger-none 

V 

Nesting-none 

Spacing: 

Horizontal  spacings  used-variable-40km  and  greater 

V 

INITIALIZATION 

Optimum  interpolation 

V 

NUMERICS 

Time  differencing: 

Leapfrog 

V 

Advection 

2nd  order  leapfrog 

V 

COORDINATE  SYSTEM 

Horizontal 

Mercator  projection 

s 

Lambert  conformal 

s 

Polar  stereographic 

Vertical: 

Terrain  following  pressure 

V 

LATERAL  BOUNDARY 
CONDITIONS 

Relaxation 

V 

TOP  BOUNDARY 
CONDITIONS 

Rigid 

V 

SURFACE 

PARAMETERIZATION  S 

Multi-level  PBL  parameterizations 

s 

SUBGRID  DIFFUSION 
PARAMETERIZATIONS 

Deformation-based  closure 

s 
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Table  3-5.  RWM  options.  (Continued) 


MODEL  PARAMETER 

OPTIONS 

USED 

DEEP  CONVECTION 
PARAMETERIZATIONS 

Kuo 

V 

RADIATION 

PARAMETERIZATIONS 

Long  and  shortwave  scheme  with  cloud  effects  and 
with  atmospheric  tendencies 

V 

RESOLVED 

CONDENSATION 

PARAMETERIZATIONS 

Precipitation  if  relative  humidity  exceeds  a  threshold 

V 

Computational  Flow  and  Relationship  between  CRAY  and  Workstation  Version 

The  model  was  run  only  on  the  workstation.  However,  not  all  data  ingestion  routines  operate  on  this 
platform  and  AF  personnel  and  AFGWC  accomplished  this  data  preparation.  These  routines  are 
associated  with  grid  dependent  quantities  and  not  the  meteorological  data. 

3.5.2  RWM  Model  Source. 

Source  code  for  RS/6000  was  received  September  1994  on  tape  from  AFGWC  with  sample  run  for -25 
April  1994. 

The  Matthews’  Vegetation  Data  was  obtained  from  the  National  Center  for  Atmospheric  Research 
(ftp.ucar.edu,  ds765.0)  and  was  downloaded  August  1994. 

The  Soil  Data  was  obtained  from  the  National  Center  for  Atmospheric  Research  (ftp.ucar.edu,  ds776.0) 
and  was  downloaded  August  1994. 

3.5.3  Execution  Speed. 

The  execution  speed  for  RWM  on  the  IBM  RS/6000  Model  370  workstation  was  about  9,900  CPU 
seconds  for  each  36  hour  forecast  (7.6%  of  real  time).  The  data  analysis  took  an  additional  10  minutes. 

3.5.4  Attributes  Noted. 

RWM  is  a  modified  version  of  the  QNGM  model  developed  by  M.  Mathor  (Mathor,  1983)  at  National 
Meteorological  Center  (NMC)  in  the  late  1970s  and  early  1980s.  There  were  two  versions  developed. 
One  with  and  one  without  the  ability  to  add  a  nested  grid.  RWM  is  the  version  without  this  capability. 
Only  recently  (within  the  last  two  years)  has  a  diurnal  cycle  and  surface  physical  parameterizations  been 
included  in  the  model.  Several  of  the  model  initialization  routines  only  run  on  a  CRAY  running  the 
COS  operating  system  and  have  not  been  ported  to  any  other  platform. 

•  Weak  points  include  : 

1 .  simplistic  parameterization  of  the  boundary  layer,  does  not  allow  for  elevated  mixed  layers, 

2.  no  microphysics  parameterization, 

3.  no  non-hydrostatic  option. 
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3.5.5  Model  Support  and  Maintenance. 

RWM  was  developed  originally  by  the  National  Meteorological  Center  as  the  QNGM,  but  was 
subsequently  abandoned  by  NMC.  AFGWC  is  the  only  user  and  developer  of  this  code.  Commercial 
support  is  unavailable. 

3.6  RAMS  EVALUATION  AND  VERIFICATION  UTILITIES  (REVU). 

The  statistical  package  we  used  for  model  intercomparison  incorporated  software  and  routines  that  are 
part  of  REVU,  making  the  assessment  of  each  atmospheric  model's  performance  possible.  REVU  was 
modified  to  incorporate  the  various  mean  quantities  required  at  each  pressure  level  and  to  develop  a 
histogram  distribution  of  the  errors. 

3.6.1  Mean  Errors. 

A  series  of  six  statistical  parameters  was  calculated.  These  quantities  measured  the  error  in  the  predicted 
field  at  an  observation  point.  The  quantities  were  summed  over  all  observations  at  a  specific  vertical 
level.  The  ten  vertical  levels  used  are  surface,  1000  mb,  850  mb,  700  mb,  500  mb,  400  mb,  300  mb,  250 
mb,  200  mb,  150  mb,  and  100  mb. 

Statistics  were  generated  at  initialization  and  at  all  forecast  times  (00,  12,  24,  and  36  hours).  The 
statistical  parameters  calculated  were: 

Root  Mean  Square  Error  (RMSE) 


(3.1) 


Mean  Absolute  Error 


EK-Sil  (3.2) 

-L=J _ 

n 


Mean  Relative  Error 


iK-*;)  (3.3) 

Oil _ 

n 
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Bias  (Mean  Normalized  Bias) 


n  fa)/  _<J)i 

X  *• '  ; 


Correlation  Coefficient 


_ Li J - 

-i=i  jL;=i  J 


Root  Mean  Square  Vector  Error  (RMSVE) 


where  O  is  a  scalar  variable,  u  and  v  are  wind  components,  n  is  the  total  number  of  i  observations, 
subscript  p  =  predicted,  subscript  o  =  observed,  and  a  bar  indicates  the  arithmetic  average  of  the  quantity. 

These  five  statistics  were  calculated  for  each  model  for  each  of  seven  model-predicted  variables.  These 
seven  variables  were: 

•  Specific  Humidity  (q) 

•  Zonal  Wind  Component  (u) 

•  Meridional  Wind  Component  (v) 

•  Temperature  (t) 

•  Sea  Level  Pressure  (sip) 

•  Geopotential  height  (z) 

•  Relative  Humidity  (rh) 
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The  amount  of  data  generated  with  this  routine  is  huge  and  will  be  available  from  DSWA  on  magnetic 
media. 

3.6.2  Distribution  Histogram. 

The  mean  data  generated  in  the  previously  described  part  of  REVU  emphasize  the  pressure  levels  and 
not  the  number  of  observations.  The  mean  quantities  also  do  not  indicate  how  often  the  required 
computational  accuracy  is  achieved.  To  provide  this  data,  REVU  was  modified  to  generate  error 
distribution  for  the  following  variables:  temperature,  surface  pressure,  dew  point  depression,  wind  speed 
under  20  knots,  wind  speed  over  20  knots,  and  wind  direction.  Each  of  these  quantities  has  an 
established  criterion.  They  are  “air  temperature  and  dew  point  depression  within  2  degrees,  wind  speed 
within  2  knots  at  speeds  less  than  20  knots  and  within  5  knots  at  speeds  greater  than  20  knots,  wind 
direction  within  30  degrees,  and  surface  pressure  within  1.7  millibars”  (SOW,  see  section  2.5.4.).  These 
criteria  were  used  to  establish  the  central  bin  boundaries.  Observation-prediction  pairs  in  the  central  bin 
met  the  AF  criteria.  These  histograms  are  based  on  the  total  number  of  observations  available  in  a  given 
set  of  runs.  Both  36  hour  forecasts  were  merged  as  were  all  forecast  times.  The  dominance  of  surface 
observations  was  precluded  by  generating  two  sets  of  histograms,  one  for  the  surface  and  one  for  all  the 
pressure  levels. 

3.7  HARDWARE. 

All  testing  was  conducted  on  one  of  two  platforms,  the  DSWA  CRAY  at  Los  Alamos  and  an  IBM, 
RS/6000  workstation  located  at  MRC,  Fort  Collins.  Personnel  from  MRC,  Santa  Barbara,  performed  the 
tests  run  on  the  CRAY.  The  following  is  a  description  of  the  hardware/compiler  and  procedures  we  used 
for  these  platforms. 

3.7.1  CRAY. 

The  DSWA  CRAY  supercomputer  used  for  the  majority  of  the  calculations  was  LANL  machine 
“gamma,”  or  specifically  gamma.lanl.gov.  This  machine  is  a  CRAY  Y-MP  with  an  extraordinary  two 
gigawords  of  memory.  During  testing  this  computer  was  subjected  to  extreme  loads  and  the  alternative 
LANL  machine  “rho”  was  used  for  some  runs.  It  is  a  CRAY  Y-MP  M64  and  runs  -20-33  percent  faster 
than  gamma. 

While  the  candidate  CWS  models  are  written  in  FORTRAN,  some  adhere  to  more  standard  syntax  rules 
than  others.  The  RAMS-3b  code  used  is  entirely  f77  code;  the  NORAPS6  code,  on  the  other  hand, 
contains  some  f90  extensions.  The  “cf77”  compiler  on  the  LANL  CRAY,  has  a  sufficient  number  of  f90 
extensions  to  compile  all  of  the  candidate  CWS  models  and  was  used. 

The  execution  time  or  CPU  provided  by  the  DSWA  facility  includes  a  term  that  is  related  to  the  amount 
of  job  swapping  that  the  machine  must  perform.  A  job  run  when  the  machine  is  nearly  idle  would  take 
less  CPU  time  than  the  same  job  run  when  the  machine  is  busy.  The  reported  CPU  times  are  average 
execution  times. 

3.7.2  IBM  RS/6000  Workstation. 

The  workstation  used  for  these  tests  is  an  EBM  RS/6000  Model  370  with  128  Mb  of  central  memory  and 
4  Gb  of  attached  disk  storage.  The  operating  system  is  IBM's  UNIX  variant,  AIX,  at  revision  level  3.2.5. 
Since  both  NORAPS6  and  MM5  use  various  CRAY-specific,  FORTRAN  constructs,  version  3.2  of  the 
AIX  FORTRAN  compiler  was  needed  to  minimize  the  number  of  code  changes.  This  compiler  is  a  full 
FORTRAN  90  implementation.  Fortunately,  many  of  the  non-standard  CRAY  extensions  to  FORTRAN 
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77  have  been  included  in  the  FORTRAN  90  standard.  The  one  extension  that  would  have  caused  the 
most  code  modifications  to  NORAPS6  and  MM5  was  the  concept  of  “automatic”  arrays.  These  are 
arrays  that  are  local  to  a  routine  (not  a  routine  argument)  that  are  dimensioned  by  arguments  passed  to 
the  routine.  The  arrays  are  automatically  dynamically  allocated  upon  entrance  to  the  routine  and 
deallocated  upon  exit  from  the  routine.  Since  FORTRAN  77  had  no  concept  of  dynamic  allocation, 
modification  of  the  codes  for  a  FORTRAN  77  compiler  would  have  been  difficult. 
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SECTION  4 
TESTS  RESULTS 


In  this  section,  the  results  of  the  various  threshold  tests,  the  data  denied  tests,  and  the  objective  tests  are 
presented.  The  amount  of  available  data  required  that  only  a  fraction  of  the  tests  could  be  presented.  In 
general,  the  forecast  maps  were  compared  at  24  hours  and  significant  differences  noted  in  the 
phenomenology  results.  Maps  for  some  of  the  differences  noted  are  provided  for  illustration.  The 
histograms  of  errors  are  provided  for  each  of  the  threshold  forecast  test  sets. 

4.1  THRESHOLD  TESTS  AND  RESULTS. 

4.1.1  SESAME  1- CONUS. 


4. 1 . 1 . 1  Objective.  This  test  was  used  to  verify  correct  operation  of  the  models  on  both  the  CRAY  and 
the  workstation  and  measure  model  performance  in  the  CONUS. 


4. 1 . 1 .2  Description.  AVE-SESAME I  (Atmospheric  Variability  Experiment — Severe  Environmental 
Storms  and  Mesoscale  Experiment)  took  place  on  10-11  April  1979.  A  vigorous  shortwave  over 
Colorado  induced  strong  cyclogenesis  centered  over  eastern  Colorado.  The  resultant  strong  surface 
warm  air  advection  over  the  lower  Midwest,  combined  with  cold  air  advection  aloft  and  a  favorable 
wind  shear  pattern,  spawned  a  major  tornado  outbreak  in  Texas  and  Oklahoma.  General  thunderstorm 
activity  was  also  observed  over  the  Midwest.  The  three  day  period  selected  was  9  Apr  79  1200  UTC 
through  12  Apr  79  1200  UTC. 


4. 1 . 1 .3  Data  Source.  NMC  gridded  data  are  contained  within  the  NCAR  data  archives  in  ds082.0,  tape 
K6838K.  The  rawinsonde  data  are  in  ds353.4,  tape  K1 101K  while  the  surface  observations  at  0  UTC 
and  12  UTC  are  in  ds464.0,  tape  K0712K. 


4. 1 . 1 .4  Phenomenological  Results  -  SESAME.  Phenomenology  results  24  hours  into  the  first  forecast 
period  ( 10  Apr  79  -  1200  UTC). 


Surface 

At  24  hours  into  the  forecast,  the  surface  front  had  progressed  down  to  the  Gulf  coast.  NORAPS6  had 
predicted  the  front  in  a  good  position  at  the  coast,  but  with  a  steeper  gradient  than  was  observed.  It  also 
had  similar  gradients  along  the  boundary  in  central  Mexico.  RWM  and  MM5  predicted  good  location 
and  strength  compared  to  the  data  analysis.  However,  RWM  produced  an  area  of  anomalous  strong 
heating  on  the  lee  of  the  Rockies  in  western  Nebraska.  The  RAMS  frontal  gradient  was  well  located,  but 
was  more  diffuse  through  Texas  and  Oklahoma  than  observed. 

RWM,  RAMS,  and  MM5  all  produced  widespread  fog  and  low-level  clouds  over  the  central  plains, 
which  was  moister  than  observed.  NORAPS6  hit  a  maximum  of  about  88  percent  relative  humidity. 

A  surface  low  of  988  mb  was  located  over  western  Colorado  at  this  time.  RAMS  had  the  low  well 
placed  with  a  central  pressure  of  982  mb,  with  the  discrepancy  possibly  due  to  differences  in  sea  level 
pressure  reduction  differences.  MM5  had  its  low  center  in  northwestern  Wyoming  at  986  mb.  RWM 
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also  had  a  986  mb  low  center,  but  in  central  Colorado.  Its  center  was  much  more  diffuse  than  observed. 
NORAPS6  had  a  999  mb  low  pressure  center  well  north  over  the  Wyoming/Montana  border.  Its  entire 
sea  level  pressure  field  demonstrated  a  high  bias  of  several  millibars.  Figures  4-1  (a  -  e)  show  the 
analysis  and  model  results  for  the  surface  pressure. 


sea  level  presstire  (mb) 


(a)  Analysis  at  24  hours 


Figure  4-1.  Surface  Pressure. 
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(b)  MM5  -  24  hour  forecast 


(c)  NORAPS6  -  24  hour  forecast 


Figure  4-1.  Surface  Pressure.  (Continued) 


(e)  RWM  -  24  hour  forecast 
Figure  4-1.  Surface  Pressure.  (Continued) 
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Upper  Air 

At  24  hours  and  500  mb,  the  observed  jet  streak  with  a  core  of  about  28  m/s  was  over  northeastern  Texas 
and  southeastern  Oklahoma  at  1200  UTC.  RAMS  predicted  the  location  of  the  core  at  a  good  location 
with  strength  of  26  m/s.  MM5  predicted  a  “noisier”  streak,  probably  due  to  vertical  momentum 
transport  in  the  convective  parameterization.  RWM  under-predicted  the  streak  with  no  well-defined 
core  and  strength  of  22  m/s  in  the  same  area.  NORAPS6  over-predicted  the  streak  with  a  maximum 
strength  of  34  m/s  and  a  location  too  far  to  the  south  over  southeastern  Texas.  Figures  4-2(a  -  e)  show 
the  analysis  and  calculated  velocity  fields  at  500  mb. 

Associated  with  this  jet  streak  was  a  shortwave  trough  with  an  axis  WNW  to  ESE  over  the  central 
plains. 

MM5  did  a  good  job  with  the  location  and  amplitude  of  the  trough.  RAMS  and  RWM  under-predicted 
the  trough  while  NORAPS6  over-predicted  amplitude  and  had  a  slower  movement  of  the  trough  than 
observed.  The  entire  geopotential  field  of  NORAPS6  also  exhibited  a  high  bias. 


(a)  Analysis  at  24  hours 


Figure  4-2.  500mb  wind  velocity. 
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(b)  MM5  -  24  hour  forecast 


(c)  NORAPS6  -  24  hour  forecast 


Figure  4-2.  500mb  wind  velocity.  (Continued) 
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(d)  RAMS  -  24  hour  forecast 


(e)  RWM  -  24  hour  forecast 
Figure  4-2.  500mb  wind  velocity.  (Continued) 
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Phenomenology  results  24  hours  into  the  second  forecast  period  (12  Apr  79  -  0000  UTC). 

Surface 

At  0000  UTC  on  12  April,  the  main  gradient  of  the  surface  front  was  located  in  central  plains,  extending 
from  Indiana  through  Illinois,  Iowa,  and  into  Nebraska.  All  models  did  a  relatively  good  job  in  the 
location  and  strength  of  the  front.  To  the  north  of  the  front,  RWM,  NORAPS6,  and  MM5  all  produced 
low  level  clouds,  while  RAMS  showed  a  moist  region,  but  not  quite  saturated,  in  the  same  location.  The 
main  difference  produced  by  the  models  was  in  their  handling  of  a  “dry  line”  in  the  southern  plains, 
extending  north  to  south  through  Oklahoma  and  Texas.  Dry  lines  are  favored  locations  for  convective 
development.  The  observed  dry  line  had  a  strong  moisture  gradient  with  relative  humilities  of  20 
percent  in  central  Texas  to  80  percent  in  eastern  Texas.  RWM  did  not  predict  the  dry  air  to  the  west, 
producing  a  gradient  of  72  percent  to  96  percent  through  Texas.  NORAPS6  produced  a  rather  diffuse 
dry  line  of  45  percent  to  72  percent  through  the  area.  MM5  predicted  a  good  location  with  a  gradient  of 
40  percent  to  90  percent.  RAMS  produced  both  good  location  and  strength  with  a  gradient  of  25  percent 
to  70  percent.  Figures  4-3(a  -  e)  show  the  analysis  and  computed  relative  humidity  fields  on  the  surface. 


relative  humidity  (pet) 


(a)  Analysis  at  24  hours 


Figure  4-3.  Surface  relative  humidity. 


(b)  MM5  -  24  hour  forecast 


(c)  NORAPS  6  -  24  hour  forecast 

Figure  4-3.  Surface  relative  humidity.  (Continued) 
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(d)  RAMS  -  24  hour  forecast 


(e)  RWM  -  24  hour  forecast 


Figure  4-3.  Surface  relative  humidity.  (Continued) 


The  surface  synoptic  flow  was  dominated  by  a  large  low-pressure  system  with  an  observed  center  of  986 
mb  in  central  Kansas.  All  models  were  slow  in  movement  of  the  center,  with  the  locations  in  western 
Kansas  and  eastern  Colorado.  NORAPS6,  as  with  the  previous  36-hour  period,  produced  a  field  that  haH 
a  high  bias  with  a  low  center  of  992  mb.  RWM  produced  a  numerically  “noisy”  field  with  a  center  of 
986  mb.  Both  RAMS  and  MM5  had  central  pressures  of  981  mb. 


Upper  Air 


The  main  feature  in  the  upper  level  flow  was  a  closed  cutoff  low  center  over  western  Nebraska  and 
northeastern  Colorado  with  a  central  geopotential  height  of  5370  m.  All  models  generally  predicted  a 
good  location.  RWM  and  MM5  predicted  too  low  a  central  height,  5340  and  5310,  respectively,  with 
RWM,  again,  producing  a  noisy  field.  NORAPS6  and  RAMS  predictions  were  not  as  low  as  the 
observed  height,  both  were  at  approximately  5400m. 


4. 1 . 1 .5  Statistical  Results.  Error  frequency  distributions  were  generated  for  temperature,  surface 
pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and -wind 
direction.  Data  from  all  forecast  times,  and  where  appropriate,  levels  were  merged.  The  central  column 
corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were  developed  to  describe  the  forecast 
results.  One  is  a  surface  set  that  includes  the  large  number  of  surface  observations  and  the  other  is  an 
upper  air  set  that  includes  the  more  sparse  upper  air  observations.  Figures  4-4(a  -  f)  are  the  surface 
results  and  Figures  4-5(a  -  e)  are  the  upper  air  results. 


4.1.2  August  1994  -  Alaska,  Korea,  Central  America,  Middle  East  Theaters. 


4. 1 .2. 1  Objective.  To  compare  the  36  hour  forecasts  from  the  candidate  models  in  a  number  of 
representative  regions  or  theaters,  data  will  be  obtained  from  the  historical  files  at  NCAR  for  August  of 
1994.  Four  regions  or  theaters  will  be  computed:  Alaska,  Central  America,  Korea,  and  the  Middle  East. 


4. 1.2.2  Description.  The  period  from  0  UTC  16  August  1994  to  0  UTC  19  August  1994  was  selected. 
From  the  500  mb  hemispheric  maps,  a  shortwave  trough  crossed  over  the  Korean  peninsula  by 
0  UTC  18  August  and  a  separate  shortwave  passed  over  and  exited  Alaska  by  0  UTC  19  August.  These 
upper  level  shortwaves  should  have  surface  frontal  systems  and  other  interesting  weather  effects 
associated  with  them.  The  same  period  was  used  for  the  Middle  East  and  Central  America.  No 
preferred  days  were  noted  for  these  regions  as  all  the  days  were  meteorologically  similar.  A  more 
interesting  period  (10-12  August),  with  similar  but  stronger  shortwaves,  was  not  selected  because  NMC 
lost  the  gridded  data  for  this  period. 


NUMBER  Of  OBSERVATIONS  I  NUMBER  OF  OBSERVATIONS 


temperature:  {predicted -observed)  model-  mm s 


TEMPERATURE  IPRE0ICTED-08SERVEDJ  MODEL.  NOHAPS 


SESAME-CONUS  DATA.  APRIL  RUNS.  SFC 
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(a)  CONUS  temperature 


Figure  4-4.  Surface  error  statistics. 
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nKuiT  occurrence  percent  occurrence 
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(b)  CONUS  pressure 


Figure  4-4.  Surface  error  statistics.  (Continued) 
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(c)  CONUS  dew  point  depression 
Figure  4-4.  Surface  error  statistics.  (Continued) 
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PERCENT  OCCURRENCE  PERCENT  OCCURRENCE 


NUMBER  OF  OBSERVATIONS 


(d)  CONUS  wind  speed  <  20  Knots 
Figure  4-4.  Surface  error  statistics.  (Continued) 
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PERCENT  OCCURRENC6  PERCENT  OCCURRENCE 


NUMBER  Of  OflSERVATIONS  I  NUMBER  OF  OBSERVATIONS 


(e)  CONUS  wind  speed  >  20  Knots 
Figure  4-4.  Surface  error  statistics.  (Continued) 
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Figure  4-4.  Surface  error  statistics.  (Continued) 
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(a)  CONUS  temperature 


Figure  4-5.  Upper  air  error  statistics. 
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(b)  CONUS  dew  point  depression 
Figure  4-5.  Upper  air  error  statistics.  (Continued) 
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Figure  4-5.  Upper  air  error  statistics.  (Continued) 
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(d)  CONUS  wind  speed  >  20  Knots 


Figure  4-5.  Upper  air  error  statistics.  (Continued) 
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(e)  CONUS  wind  direction 
Figure  4-5.  Upper  air  error  statistics.  (Continued) 
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PERCENT  OCCURRENCE  PERCENT  OCCURRENCE 


The  Hata  analysis  is  not  as  clear  and  definitive  as  it  was  in  the  CONUS  case,  since  the  analysis  is  a 
mixture  of  heavily  smoothed  global  fields  and  observations  where  available.  These  result  from  the  large 
data-void  regions  of  the  four  theaters.  Therefore,  interpretation  of  the  analysis  and  the  comparison  with 
the  model  results  will  be,  by  necessity,  somewhat  less  conclusive  than  in  the  CONUS  case. 

4. 1.2.3  Data  Source.  The  NMC  gridded  data  are  contained  within  the  NCAR  data  archives  in  ds082.0, 
tape  21810.  The  rawinsonde  data  are  in  ds353.1,  tape  Y22553  and  the  surface  observations  at  0  UTC 
and  12  UTC  are  in  ds464.0,  tape  Y22551. 

4. 1.2.4  Phenomenological  Results  -  Alaska.  Phenomenology  results  at  24  hours  into  the  first  forecast 
period  (17  Aug  94  -  0000  UTC). 


Surface 

The  surface  wind  field  showed  a  low-level  jet  in  advance  of  a  surface  low,  centered  in  the  western  part 
of  the  domain.  The  analyzed  jet  maximum  was  up  to  17  m/s  off  the  western  Alaska  coast. 

RAMS  and  MM5  predicted  good  speed  of  the  jet  and  showed  a  slowing  of  the  wind  over  land  as  the 
analysis  because  of  vegetation  parameterizations.  RWM  over-predicted  the  speed  (up  to  23  m/s),  while 
NORAPS6  under-predicted  the  intensity  at  15  m/s  and  showed  no  velocity  gradients  at  the  coastline 
since  it  does  not  include  a  land-use  parameterization. 

RAMS  and-NORAPS6  generally  predicted  a  good  temperature  field.  RWM  generated  a  strong  front  in 
the  north  Pacific  that  was  not  in  the  data  analysis,  while  MM5  generated  a  rather  strange  localized 
perturbation  on  the  Aleutian  peninsula.  The  analysis  was  very  moist  at  the  surface  over  most  ocean 
areas.  RAMS  and  MM5  predicted  saturated  conditions  over  most  water  (which  could  be  interpreted  as 
low  stratus  clouds).  NORAPS6  generated  some  saturated  regions  west  and  north  of  Alaska,  but  showed 
significant  boundary  problems,  as  NORAPS  6  does  not  use  the  humidity  observations  to  initialize  the 
forecast.  RWM  suffered  from  a  bad  initial  analysis,  and  did  not  recover.  Figures  4-6(a  -  e)  depict  the 
surface  winds  at  24  hours. 

The  surface  low  was  located  west  of  Alaska  at  this  time  in  the  analysis  with  a  central  pressure  of  996 
mb.  All  models  predicted  a  good  location.  RAMS  and  RWM  under-predicted  the  pressure  at  992  mb, 
with  RWM  continuing  its  feature  of  producing  noisy  pressure  and  geopotential  fields.  MM5  predicted 
the  low  center  at  996  mb  and  NORAPS6  predicted  998  mb. 
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(d)  RAMS  -  17  Aug  94  -  0000  UTC  forecast 

Figure  4-6.  Surface  winds  -  Alaska.  (Continued) 
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(e)  RWM  -  17  Aug  94  -  0000  UTC  forecast 


Figure  4-6.  Surface  winds  -  Alaska.  (Continued) 


Upper  Air 

At  500  mb,  a  jet  streak  dominated  the  domain  with  a  core  maximum  speed  of  38  m/s.  All  models  had 
generally  a  good  location  and  speed.  The  geopotential  field  had  a  trough  west  of  Alaska  and  a  ridge  to 
the  south.  All  did  well,  although  RWM  was  again,  numerically  noisy. 

Phenomenology  results  at  24  hours  into  the  second  forecast  period  (18  Aug  94  -1200  UTC). 

Surface 

The  analysis  showed  that  the  low  center  had  moved  over  the  northern  Alaska  coast  with  a  central 
pressure  of  994  mb,  although  the  circulation  had  weakened  from  the  past  36  hours.  All  models  under¬ 
predicted  the  intensity  of  the  circulation  with  NORAPS6  at  999  mb,  MM5  at  998  mb,  RAMS  at  1000 
mb,  and  RWM  too  noisy  to  distinguish.  Figure  4-7(a  -  e)  show  the  predicted  surface  pressure  fields. 

MM5  and  RAMS  were  still  near  saturated  over  water.  NORAPS 6  predicted  many  gradients  that  were 
not  present  in  the  data  analysis.  RWM,  again,  had  problems  because  of  the  initial  analysis. 
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sea  level  pressure  (mb) 


(a)  Analysis  at  18  Aug  94  -  1200  UTC 


(b)  MM5  -  18  Aug  94  -  1200  UTC  forecast 
Figure  4-7.  Surface  pressure. 
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(c)  NORAPS6  -  18  Aug  94  -  1200  UTC  forecast 


(d)  RAMS  -  18  Aug  94  -  1200  UTC  forecast 


Figure  4-7.  Surface  pressure.  (Continued) 


(e)  RWM  -  18  Aug  94  -  1200  UTC  forecast 


Figure  4-7.  Surface  pressure.  (Continued) 

Upper  Air 

As  with  surface  low,  all  models  under-predicted  the  intensity  of  the  500  mb  trough  over  north  Alaska. 

An  analyzed  jet  core  of  40  m/s  was  associated  with  this  trough.  RWM  produced  a  noisy,  misplaced  core 
with  36  m/s  maximum  speed.  NORAPS6  over-predicted  the  speed  at  48  m/s  and  placed  the  core  too  far 
to  the  northeast.  RAMS  and  MM5  produced  a  good  location,  with  RAMS  slightly  under  at  38  and  MM5 
slightly  over  at  44  m/s. 

4. 1.2.5  Statistical  Results  -  Alaska.  Error  frequency  distributions  were  generated  for  temperature, 
surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and 
wind  direction.  Data  from  all  forecast  times,  both  August  forecasts,  and  where  appropriate,  levels  were 
merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were 
developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of  surface 
observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air  observations. 
Figures  4-8(a  -  f)  are  the  surface  results  and  Figures  4-9(a  -  e)  are  the  upper  air  results. 
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NUMBER  OF  OBSERVATIONS 


(b)  August  -  Alaska  pressure 
Figure  4-8.  Surface  error  statistics.  (Continued) 
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(c)  August  -  Alaska  dew  point  depression 
Figure  4-8.  Surface  error  statistics.  (Continued) 
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(d)  August  -  Alaska  wind  speed  <  20  Knots 


Figure  4-8.  Surface  error  statistics.  (Continued) 
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(e)  August  -  Alaska  wind  speed  >  20  Knots 
Figure  4-8.  Surface  error  statistics.  (Continued) 
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(f)  August  -  Alaska  wind  direction 


Figure  4-8.  Surface  error  statistics.  (Continued) 
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(b)  August  -  Alaska  dew  point  depression 


Figure  4-9.  Upper  air  error  statistics.  (Continued) 
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HUMBER  Of  OBSERVATIONS  NUMBER  OF  OBSERVATIONS 


(c)  August  -  Alaska  wind  speed  <  20  Knots 
Figure  4-9.  Upper  air  error  statistics.  (Continued) 
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(d)  August  -  Alaska  wind  speed  >  20  Knots 


Figure  4-9.  Upper  air  error  statistics.  (Continued) 
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(e)  August  -  Alaska  wind  direction 
Figure  4-9.  Upper  air  error  statistics.  (Continued) 
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4. 1.2.6  Phenomenological  Results  -  Korea.  Phenomenology  results  at  24  hours  into  the  first  forecast 
period  ( 1 7  Aug  94  -  0000  UTC). 

Surface 

The  surface  wind  analysis  was  dominated  by  easterlies  south  of  Japan,  while  a  jet  from  the  southwest 
was  located  over  the  Sea  of  Japan  starting  just  northeast  of  the  Korean  peninsula  extending  along  west 
coast  of  Japan.  RAMS  did  very  well  with  good  speed  and  direction  of  both  flow  branches.  NORAPS6 
did  very  poorly.  MM5  had  good  flow  near  Korea,  but  it  cut  off  the  jet  with  a  southeasterly  flow 
emanating  from  the  boundary.  RWM  did  fairly  well  with  the  direction,  but  over-predicted  the  maximum 
speed.  Figures  4-10(a  -  e)  show  the  surface  wind.  RAMS  and  MM5  produced  near  saturated  over  the 
water  areas  east  and  west  of  Korea.  The  analysis  was  also  moist  with  the  relative  humidity  analyzed  at 
90-95  percent.  RAMS  forecast  an  anomalous  high  region  of  relative  humidity  in  the  northwest  comer  of 
the  domain.  NORAPS6  and  RWM  produced  large  boundary  gradients  due  to  the  initial  analysis. 


speed  (m/s) 


(a)  Analysis  at  17  Aug  94  -  0000  UTC 
Figure  4-10.  Surface  wind  -  Korea. 
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(b)  MM5  -  17  Aug  94  -  0000  UTC  forecast 


(c)  NORAPS6  -  17  Aug  94  -  0000  UTC  forecast 


Figure  4-10.  Surface  wind  -  Korea.  (Continued) 


(d)  RAMS  -  17  Aug  94  -  0000  UTC  forecast 


(e)  RWM  -  17  Aug  94  -  0000  UTC  forecast 
Figure  4-10.  Surface  wind  -  Korea.  (Continued) 
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Upper  Air 

The  analysis  showed  a  weak  surface  front  along  the  East  Asian  coast,  extending  from  the  low  centered 
near  the  northern  boundary.  RAMS  predicted  a  good  location  and  strength  for  the  front.  MM5  over¬ 
developed  the  front  and  developed  a  new  low  center  west-southwest  of  Korea  that  was  not  in  the  data 
analysis.  NORAPS6  had  a  poor  orientation  of  the  northern  portion  of  the  front  and  had  a  high-pressure 
bias  in  the  entire  domain.  RWM  was  again  noisy  and  produced  a  too  strong  low.  A  jet  streak,  with  a 
core  speed  of  30  m/s,  extended  from  the  southwest  to  the  northeast  of  Korea  in  the  data  analysis.  All 
models  had  a  fair  location  and  speed.  RAMS  and  MM5  did  a  good  job  capturing  the  analyzed  trough  in 
the  northern  part  of  the  domain  and  the  ridge  over  Japan.  NORAPS6  also  did  fairly  well,  while  RWM 
continued  to  be  noisy. 

Phenomenology  results  at  24  hours  into  the  second  forecast  period  (18  Aug  94  -  1200  UTC). 

Surface 

At  this  time,  high  pressure  dominated  the  Korean  region,  with  a  northeasterly  flow  in  the  Sea  of  Japan, 
east  of  Korea,  while  the  winds  turned  anticyclonicly  south  and  west  of  Korea.  NORAPS6  and  RWM 
forecast  a  fairly  good  direction,  while  MM5  produced  easterly  winds  over  the  Asian  continent  instead  of 
southerly.  RAMS  produced  a  southwesterly  rather  than  northeasterly  flow  in  the  Sea  of  Japan.  This  is 
seen  in  Figures  4-1  l(a  -  e). 


(a)  Analysis  at  18  Aug  94  -  1200  UTC 


Figure  4-11 .  Surface  wind  -  Korea. 
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(b)  MM5  -  18  Aug  94  -  1200  UTC  forecast 


(c)  NORAPS6  -  18  Aug  94  -  1200  UTC  forecast 


Figure  4-11.  Surface  wind  -  Korea.  (Continued) 


(e)  RWM  -  18  Aug  94  -  1200  UTC  forecast 

Figure  4-11.  Surface  wind  -  Korea.  (Continued) 
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The  data  analysis  showed  a  large  low-pressure  system  moving  in  from  the  west  boundary.  All  models 
captured  the  entry  of  this  system,  but  RAMS  and  NORAPS6  over-predicted  a  low  development  near  the 
northeastern  comer  of  the  domain.  MM5  produced  a  fictitious  low  in  the  southwestern  part  of  the 
domain. 


Upper  Air 

The  500  mb  analysis  showed  an  upper  level  ridge  over  the  western  part  of  the  domain  and  a  trough  over 
the  eastern  part.  All  models  did  a  fairly  good  job,  but  under-predicted  the  ridge  intensity  over  the  China 
mainland. 

4. 1.2.7  Statistical  Results  -  Korea.  Error  frequency  distributions  were  generated  for  temperature, 
surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and 
wind  direction.  Data  from  all  forecast  times,  both  August  forecasts,  and  where  appropriate,  levels  were 
merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were 
developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of  surface 
observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air  observations. 
Figures  4-12(a  -  f)  are  the  surface  results  and  Figures  4-13(a  -  e)  are  the  upper  air  results. 


NUMBER  Of  OBSERVATIONS  1  NUMBER  OF  OBSERVAT  IONS 


KOREA  DATA.  AUGUST  RUNS.  SFC 


KOREA  DATA.  AUGUST  RUNS.  SFC 


TEMPERATURE  I  PREDICTED-OBSERVED  I  MODEL.  RAMS 


TEMPERATURE  IPREDICTED-C8SERVE0 J  MODEL.  RVM 


(a)  Temperature 

Figure  4-12.  Surface  error  statistics,  August  -  Korea. 
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(b)  Pressure 

Figure  4-12.  Surface  error  statistics,  August  -  Korea.  (Continued) 
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(c)  Dew  point  depression 


Figure  4-12.  Surface  error  statistics,  August  -  Korea.  (Continued) 
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(d)  Wind  speed  <  20  Knots 

Figure  4-12.  Surface  error  statistics,  August  -  Korea.  (Continued) 


94 


PERCENT  OCCURRENCE  1  PERCENT  OCCURRENCE 


MUNICH  or  OMEAmiOHS  HUMBER  Of  OBSERVATIONS 


(e)  Wind  speed  >  20  Knots 

Figure  4-12.  Surface  error  statistics,  August  -  Korea.  (Continued) 
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Figure  4-13.  Upper  air  error  statistics,  August  -  Korea. 
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(b)  Dew  point  depression 


Figure  4-13.  Upper  air  error  statistics,  August  -  Korea.  (Continued) 
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Figure  4-13.  Upper  air  error  statistics,  August  -  Korea.  (Continued) 
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(d)  Wind  speed  >  20  Knots 

Figure  4-13.  Upper  air  error  statistics,  August  -  Korea.  (Continued) 
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4. 1.2.8  Phenomenological  Results  -  Middle  East.  Phenomenology  results  at  24  hours  into  the  first 
forecast  period  (17  Aug  94  -  0000  UTC). 

Surface 

The  surface  sea  level  pressure  analysis  showed  a  “heat  iow”  with  a  broad  center  over  much  of  the 
Persian  Gulf/Saudi  Arabian  region  with  localized  areas  of  992  mb  pressure.  All  models  captured  the 
general  pattern  and  all  also  had  the  general  pattern  of  cooling  over  Turkey  and  Iran,  with  higher 
temperatures  over  Saudi  Arabia.  All  models  were  moister  than  the  data  analysis,  although  RAMS  was 
driest. 

Upper  Air 

The  500  mb  data  analysis  showed  an  anticyclone  over  Saudi  Arabia  and  the  Red  Sea.  MM5  predicted  a 
trough,  while  all  others  had  an  anticyclone  that  was  slightly  weaker  than  the  analysis. 

Phenomenology  results  at  24  hours  into  the  second  forecast  period  (17  Aug  94  -  0000  UTC). 

Surface 

The  most  significant  feature  in  the  surface  analysis  was  a  “heat  low”  elongated  from  the  northwest  to  the 
southeast  over  Iran  of  986  mb  central  pressure.  RAMS  captured  this  feature  very  well,  forecasting  a 
good  orientation  and  central  pressure  of  986  mb.  MM5  forecast  a  weak  center  of  996  mb  and  shifted 
south  over  Saudi  Arabia.  It  forecast  a  1010  mb  localized  high  over  Iran.  NORAPS6  and  RWM  also 
show  a  ridge  over  Iran.  RAMS  also  did  very  well  with  the  relative  humidity  forecast;  all  other  models 
were  too  high  over  the  desert  region  for  the  August  period.  In  conjunction  with  this,  RAMS  did  very 
well  with  the  temperature  forecasts  while  the  others  did  not  heat  enough  during  the  day.  Figures 
4-14(a  -  e)  show  the  surface  pressure. 
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(c)  NORAPS  6  -  18  Aug  94-  1200  UTC  forecast 


(d)  RAMS-  18  Aug  94  -  1200  UTC  forecast 


Figure  4-14.  Surface  pressure  -  Middle  East.  (Continued) 
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(e)  RWM  -  18  Aug  94  -  1200  UTC  forecast 


Figure  4-14.  Surface  pressure  -  Middle  East.  (Continued) 

Upper  Air 

The  500  mb  analysis  showed  an  anticyclone  in  southern  part  of  the  domain  with  weak  trough  in  the 
north.  MM5  did  not  forecast  the  anticyclone,  while  all  other  models  did.  All  models  forecast  the  trough 
to  one  extent  or  another,  although  the  data  analysis  was  difficult  to  interpret. 


4- 1  -2.9  Statistical  Results  Middle  East.  Error  frequency  distributions  were  generated  for  temperature, 
surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and 
wind  direction.  Data  from  all  forecast  times,  both  August  forecasts,  and  where  appropriate,  levels  were 
merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were 
developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of  surface 
observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air  observations. 
Figures  4-15(a  -  f)  are  the  surface  results  and  Figures  4-16(a  -  e)  are  the  upper  air  results. 
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(a)  Temperature 

Figure  4-15.  Surface  error  statistics,  August  -  Middle  East. 
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(b)  Pressure 

Figure  4-15.  Surface  error  statistics,  August  -  Middle  East.  (Continued) 
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(c)  Dew  point  depression 


Figure  4-15.  Surface  error  statistics,  August  -  Middle  East.  (Continued) 
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(d)  Wind  speed  <  20  Knots 

Figure  4-15.  Surface  error  statistics,  August  -  Middle  East.  (Continued) 
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(e)  Wind  speed  >  20  Knots 

Figure  4-15.  Surface  error  statistics,  August  -  Middle  East.  (Continued) 
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(f)  Wind  direction 

Figure  4-15.  Surface  error  statistics,  August  -  Middle  East.  (Continued) 


111 


PERCENT  OCCURRENCE  PERCENT  OCCURRENCE 


WKBER  Of  O0SCRYAT IONS  NUMBER  OF  OBSERVATIONS 


(a)  Temperature 


Figure  4-16.  Upper  air  error  statistics,  August  -  Middle  East. 
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Figure  4-16.  Upper  air  error  statistics,  August  -  Middle  East.  (Continued) 
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Figure  4-16.  Upper  air  error  statistics,  August  -  Middle  East.  (Continued) 
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Figure  4-16.  Upper  air  error  statistics,  August  -  Middle  East.  (Continued) 
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Figure  4-16.  Upper  air  error  statistics,  August  -  Middle  East.  (Continued) 
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4.1.2.10  Phenomenological  Results  -  Central  America.  Phenomenology  results  at  24  hours  into  the  first 
forecast  period  (17  Aug  94  -  0000  UTC). 

Surface 

This  domain  had  the  least  amount  of  observation  than  any  of  the  regions.  RAMS,  MM5,  and  RWM 
showed  evidence  of  a  sea  breeze  convergence  zone  on  the  southern  coast  of  Central  America. 

NORAPS6  did  not,  probably  due  to  the  lack  of  a  land-use  parameterization.  Temperatures  generally 
looked  reasonable  in  RAMS,  MM5,  and  RWM.  MM5  looked  too  moist  over  the  water  areas,  while 
NORAPS6  and  RWM  seemed  too  dry. 

Upper  Air 

All  models  had  the  500  mb  flow  reasonably  correct.  MM5  had  strong  perturbation  due  to  convective 
scheme  over  Haiti.  The  RAMS  500  mb  height  field  looked  good  along  with  MM5,  although  MM5 
demonstrated  a  few  boundary  perturbations.  NORAPS 6  seemed  to  have  incorrect  gradients  near  the 
northern  boundary,  while  RWM  was  noisy  but  seemed  to  have  reasonable  values. 

Phenomenology  results  at  24  hours  into  the  second  forecast  period  (17  Aug  94  -  0000  UTC). 

Surface 

NORAPS6  cooled  more  than  the  others  did  over  the  land  areas.  RAMS  was  still  reasonable  on  the 
relative  humidity  forecast,  while  MM5  continued  too  wet  and  NORAPS6  and  RWM  too  dry.  MM5  and 
RWM  forecast  a  low  development  north  of  Panama,  which  was  not  in  the  data  analysis.  RAMS  and 
NORAPS6  did  not  reproduce  the  low.  RAMS  had  a  high  sea  level  pressure  bias  in  this  run. 

Upper  Air 

The  500  mb  wind  analysis  showed  an  easterly  jet  core  centered  about  300  km  north  of  Panama  with  a 
maximum  speed  of  16  m/s.  RAMS  forecast  the  core  slightly  weaker  at  13  m/s  and  too  far  east.  MM5 
produced  a  more  diffuse  core  with  localized  areas  up  to  20  m/s.  NORAPS6  placed  the  core  too  far  to  the 
northwest  with  a  maximum  speed  of  14  m/s.  RWM  had  a  good  location  with  a  maximum  speed  of  15 
m/s.  All  models  produced  reasonably  similar  500  mb  height  patterns,  with  RAMS  having  a  high  bias 
and  RWM  continuing  to  be  noisy.  The  500  mb  winds  are  shown  in  Figures  4-17(a  -  e). 
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speed  (m/s) 


(a)  Analysis  at  18  Aug  94  -  1200  UTC 


(b)  MM 5  -  18  Aug  94  -  1200  UTC  forecast 
Figure  4-17.  500mb  winds  -  Central  America. 
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(c)  NORAPS6  -  18  Aug  94  -  1200  UTC  forecast 


(d)  RAMS  -  18  Aug  94  -  1200  UTC  forecast 
Figure  4-17.  500mb  winds  -  Central  America.  (Continued) 
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(e)  RWM  -  18  Aug  94  -  1200  UTC  forecast 


Figure  4-17.  500mb  winds  -  Central  America.  (Continued) 


4. 1.2.11  Statistical  Results  Central  America.  Error  frequency  distributions  were  generated  for 
temperature,  surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20 
knots,  and  wind  direction.  Data  from  all  forecast  times,  both  August  forecasts,  and  where  appropriate, 
levels  were  merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures 
were  developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of 
surface  observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air 
observations.  Figures  4-18(a  -  f)  are  the  surface  results  and  Figures  4-19(a  -  e)  are  the  upper  air  results. 


4.1.3  November  1994  -  Alaska,  Korea,  Middle  East,  Central  America  Theaters. 


4. 1 .3. 1  Objective.  To  compare  36  hour  forecasts  from  the  candidate  models  in  a  four  representative 
regions  or  theaters.  This  information  will  supplement  that  developed  during  the  previous  forecasts  using 
data  obtained  from  the  AF.  These  runs  were  to  emulate  an  operational  setting.  The  data  was  obtained 
from  the  AFGWC  in  a  fashion  analogous  to  the  way  it  would  be  obtained  in  an  operational  situation.  A 
specific  cut  off  time  was  established  and  only  data  and  corrections  received  before  this  time  were 
included. 
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Figure  4-18.  Surface  error  statistics,  August  -  Central  America.  (Continued) 
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Figure  4-18.  Surface  error  statistics,  August  -  Central  America.  (Continued) 
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Figure  4-18.  Surface  error  statistics,  August  -  Central  America. 


(Continued) 
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Figure  4-18.  Surface  error  statistics,  August  -  Central  America.  (Continued) 
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Figure  4-19.  Upper  air  error  statistics,  August  -  Central  America. 
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Figure  4-19.  Upper  air  error  statistics,  August  -  Central  America.  (Continued) 
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(c)  Wind  speed  <  20  Knots 
Figure  4-19.  Upper  air  error  statistics,  August  -  Central  America.  (Continued) 
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(d)  Wind  speed  >  20  Knots 


Figure  4-19.  Upper  air  error  statistics,  August  -  Central  America.  (Continued) 
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(e)  Wind  direction 

Figure  4-19.  Upper  air  error  statistics,  August  -  Central  America.  (Continued) 
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4. 1 .3.2  Description.  Surface  and  500  mb  maps  were  produced  from  the  GSM  gridded  data  files  for 
1200Z  14  Nov  1994.  These  maps  indicated  that  a  strong  trough  had  passed  over  Korea,  leaving  a 
prominent  high-pressure  system  with  considerable  thermal  contrast  across  the  peninsula.  An  impressive 
surface  low  was  present  just  off  the  Alaska  coast,  and  a  strong  temperature  gradient  existed  across  the 
state.  A  surface  front  extended  from  Nebraska  northeastward  into  the  upper  Midwest,  with  an 
approaching  trough  from  the  intermountain  region.  Relatively  benign  conditions  existed  over  the 
Middle  East  with  the  closest  weather  feature  being  a  broad  trough  in  the  proximity  of  Turkey.  The 
weather  highlight  for  these  five  geographic  areas  is  a  strong  cyclone  off  the  north  coast  of  Cuba,  which 
produces  mainly  northeast  flow  aloft  over  the  Central  American  theater. 

The  desire  to  emulate  an  operational  forecast  resulted  in  some  minor  changes  in  the  way  the  codes  were 
operated.  The  August  runs  mirrored  the  typical  procedures  used  to  compare  historical  forecasts.  An 
analysis  of  the  observations  and  global  model  output  were  used  each  twelve  hour  period  to  provide  the 
outer  edge  boundary  conditions.  However,  the  November  runs  were  accomplished  using  only  the  global 
model  output  to  provide  the  boundary  conditions.  Code  initialization  in  both  August  and  November 
were  identical.  The  analysis  charts  that  are  provided  in  this  section  for  comparison  to  the  candidate 
model’s  output  only  contain  GSM  data.  The  phenomenology  comparisons  are  therefore  against  the 
GSM  output,  which  is  always  smoother  because  of  its  coarser  output. 

The  fact  that  any  of  the  TFM-candidate  models  would  differ  from  the  GSM  forecasts  should  not 
necessarily  be  taken  as  poor  performance;  in  fact,  such  differences  should  be  expected  because  of  the 
much  higher  resolution  used  for  the  candidate  models.  A  side  effect  is  increased  boundary  “noise”  since 
the  candidate  model  boundaries  are  forced  to  match  the  smoother  GSM  fields  at  the  boundary.  The' 
statistical  data  was  generated  using  the  actual  observations  at  the  forecast  times. 

4.13.3  Data  Source.  All  data  was  supplied  by  AFGWC  and  consisted  of  GSM  analysis  at  the  initial 
time  for  each  36  hour  run  (14  Nov.  1200  UTC  and  16  Nov.  0000  UTC),  rawinsonde  and  surface 
observations  from  those  times,  and  forecast  data  at  12  hour  intervals  from  the  GSM  forecast  runs. 
Surface  observation  and  rawinsonde  data  at  the  forecast  times  was  also  obtained  from  the  same  source 
and  used  for  statistical  data  preparation. 

Note:  The  observational  data  received  from  AFGWC  had  far  fewer  stations  reporting  in  the  vast 
majority  of  data  times  as  compared  to  the  data  archived  at  NCAR.  This  is  probably  a  reflection  of  the 
data  cut  off  time  used. 

4.13.4  Phenomenological  Results  -  Alaska.  Phenomenology  results  at  24  hours  into  the  first  forecast 
period  (15  Nov  94  -  1200  UTC). 

Surface 

GSM  forecast  a  972  mb  surface  low  over  west  central  Alaska  at  this  time.  RAMS,  MM5,  and 
NORAPS6  predicted  the  center  farther  to  the  northwest  with  a  central  pressure  of  975  mb  for  RAMS, 
981  for  MM5  and  981  forNORAPS6.  RWM’s  prediction  was  very  different  from  the  rest,  with  a  very 
elongated  center  extending  up  to  the  northeast  Alaska  coast.  Its  field  lacked  the  distinct  closed  center 
present  in  the  other  4  models. 

Upper  Air 

The  500  mb  fields  were  dominated  by  an  intense  short-wave  supporting  the  surface  low.  All  models 
produced  similar  fields,  except  for  RWM,  which  moved  the  wave  faster  than  the  others. 
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Phenomenology  results  at  24  hours  into  the  second  forecast  period  (17  Nov  94  -  0000  UTC). 

Surface 

Thirty-six  hours  later,  GSM  predicted  the  surface  low  to  move  over  the  northern  Yukon/Northwest 
territories  and  weakening  with  a  990  mb  central  pressure.  It  also  developed  a  frontal  wave  over  the 
ocean  to  the  southwest  of  Alaska  with  a  1000  mb  pressure.  RAMS,  MM5,  and  NORAPS6  all  favored 
the  development  of  the  southern  system,  further  to  the  east  than  GSM,  with  NORAPS6  producing  a  1000 
mb  central  pressure,  999  for  MM5,  and  994  for  RAMS.  RWM  produced  a  much  weaker  system, 
somewhat  similar  to  GSM.  The  northern  low  was  weakened  much  more  by  the  regional  models  with 
central  pressures  between  998  mb  and  1001  mb.  RWM  was  weakest  of  all  at  1006  mb.  (Figures 
4-20(a  -  e). 


sea  level  pressure  (mb). 


(a)  GSM  Analysis  at  17  Nov  94  -  0000  UTC 
Figure  4-20.  Surface  pressure  -  Alaska. 
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(b)  MM5-  17  Nov  94  -  0000  UTC  forecast 


(c)  NORAPS 6  -  17  Nov  94  -  0000  UTC  forecast 


Figure  4-20.  Surface  pressure  -  Alaska.  (Continued) 
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(d)  RAMS  -  17  Nov  94  -  0000  UTC  forecast 


(e)  RWM  -  17  Nov  94  -  0000  UTC  forecast 
Figure  4-20.  Surface  pressure  -  Alaska.  (Continued) 
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Upper  Air 

The  500  mb  wind  field  displayed  a  general  ridging  in  the  southern  part  of  the  domain.  The  forecasts 
were  similar  for  all  models,  except  for  RWM,  which  moved  the  jet  core  much  further  east  than  the  others 
were. 

4. 1.3.5  Statistical  Results  -  Alaska.  Error  frequency  distributions  were  generated  for  temperature, 
surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and 
wind  direction.  Data  from  all  forecast  times,  both  November  forecasts,  and  where  appropriate,  levels 
were  merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were 
developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of  surface 
observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air  observations. 

Figures  4-21  (a  -  f)  are  the  surface  results  and  Figures  4-22(a  -  e)  are  the  upper  air  results. 
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(a)  Temperature 

Figure  4-21.  Surface  error  statistics,  November  -  Alaska. 
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Figure  4-21.  Surface  error  statistics,  November  -  Alaska.  (Continued) 
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(d)  Wind  speed  <  20  Knots 

Figure  4-21.  Surface  error  statistics,  November  -  Alaska.  (Continued) 
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Figure  4-21.  Surface  error  statistics,  November  -  Alaska.  (Continued) 
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Figure  4-21.  Surface  error  statistics,  November  -  Alaska.  (Continued) 
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Figure  4-22.  Upper  air  error  statistics,  November  -  Alaska. 
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(b)  Dew  point  depression 


Figure  4-22.  Upper  air  error  statistics,  November  -  Alaska.  (Continued) 
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Figure  4-22.  Upper  air  error  statistics,  November  -  Alaska.  (Continued) 
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Figure  4-22.  Upper  air  error  statistics,  November  -  Alaska.  (Continued) 
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Figure  4-22.  Upper  air  error  statistics,  November  -  Alaska.  (Continued) 
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4. 1.3.6  Phenomenological  Results  -  Korea.  Phenomenology  results  at  24  hours  into  the  first  forecast 
period  (15  Nov.  94  -  1200  UTC). 

Surface 

The  GSM  forecast  at  this  time  was  dominated  by  a  surface  high  of  1052  mb  central  pressure  centered  to 
the  northeast  of  Korea.  All  models  showed  a  similar  forecast.  The  main  surface  thermal  gradient  in  the 
regional  models  was  located  roughly  coincident  with  the  coastlines  (due  to  the  warmer  waters  of  the  Sea 
of  Japan),  while  GSM  showed  very  little  effect  of  the  coast  (because  of  its  coarser  resolution)  and  had 
the  front  well  south  of  Japan. 

Upper  Air 

At  500  mb,  all  5  models  showed  the  same  general  ridging  pattern  over  the  region. 

Phenomenology  results  at  24  hours  into  the  first  forecast  period  (17  Nov  94  -  0000  UTC). 

Surface 

At  this  time,  the  models  moved  the  main  high-pressure  center  to  the  east  of  Japan  and  left  behind  a 
secondary  high  over  China.  The  main  differences  among  the  models  concerned  this  secondary  high. 
GSM  and  RAMS  located  the  high  to  the  west-southwest  of  Korea  with  about  a  1034  mb  center. 
NORAPS6  had  the  high  in  approximately  the  same  place  but  with  a  1039  mb  center,  while  MM5  located 
the  high  well  to  the  south  with  a  1037  mb  center.  RWM,  although  generally  controlling  its  numerical 
noise  better  with  the  GSM  boundary  conditions,  was  too  noisy  to  interpret  in  this  case.  The  surface 
thermal  gradients  showed  similar  characteristics  to  the  previous  36  hours.  The  nighttime  cooling  was 
different  among  the  models  with  RAMS,  MM5,  and  NORAPS6  cooling  down  to  about  -14  C  while 
GSM  and  RWM  cooled  to  only  about  -4  C.  Figures  4-23(a  -  e)  show  the  predicted  surface  pressure. 

Upper  Air 

The  maximum  winds  over  the  Korean  region  at  500  mb  were  predicted  by  MM5,  NORAPS6,  and  RWM 
to  be  about  22  m/s  and  located  directly  over  the  Korean  peninsula.  GSM  and  RAMS  had  the  maximum 
winds  located  to  the  north  of  the  peninsula  at  20  m/s.  A  500  mb  trough  was  moving  in  from  the  western 
boundary  over  the  Chinese  mainland.  All  models  handled  this  similarly. 
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(c)  N0RAPS6  -  17  Nov  94  -  0000  UTC  forecast 


(d)  RAMS-  17  Nov  94  -  0000  UTC  forecast 


Figure  4-23.  Surface  pressure  -  Korea.  (Continued) 
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(e)  RWM  -  17  Nov  94  -  0000  UTC  forecast 


Figure  4-23.  Surface  pressure  -  Korea.  (Continued) 

4. 1.3.7  .Statistical  Results  -  Korea.  Error  frequency  distributions  were  generated  for  temperature, 
surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and 
wind  direction.  Data  from  all  forecast  times,  both  November  forecasts,  and  where  appropriate,  levels 
were  merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were 
developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of  surface 
observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air  observations. 
Figures  4-24(a  -  f)  are  the  surface  results  and  Figures  4-25(a  -  e)  are  the  upper  air  results. 

4. 1.3.8  Phenomenological  Results  -  Middle  East.  Phenomenology  results  at  24  hours  into  the  first 
forecast  period  (15  Nov.  94  -  1200  UTC). 

Surface 

The  surface  pressure  field  at  this  time  was  a  rather  confused  field  with  several  small-scale  features 
handled  differently  by  all  models.  The  consistent  features  included  a  weak  surface  low  over  the  eastern 
Mediterranean  Sea  and  a  very  small  scale  low  to  the  east  of  the  Black  Sea.  RAMS  produced  a  heat  low 
over  Saudi  Arabia  and  a  localized  region  of  high  pressure  (due  to  precipitation)  over  Iran. 
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(a)  Temperature 

Figure  4-24.  Surface  error  statistics,  November  -  Korea. 
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(b)  Pressure 

Figure  4-24.  Surface  error  statistics,  November  -  Korea.  (Continued) 
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Figure  4-24.  Surface  error  statistics,  November  -  Korea.  (Continued) 
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Figure  4-24.  Surface  error  statistics,  November  -  Korea.  (Continued) 
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Figure  4-24.  Surface  error  statistics,  November  -  Korea.  (Continued) 
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Figure  4-24.  Surface  error  statistics,  November  -  Korea.  (Continued) 
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Figure  4-25.  Upper  air  error  statistics,  November  -  Korea. 
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(b)  Dew  point  depression 

Figure  4-25.  Upper  air  error  statistics,  November  -  Korea.  (Continued) 
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(c)  Wind  speed  <  20  Knots 

Figure  4-25.  Upper  air  error  statistics,  November  -  Korea.  (Continued) 
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Figure  4-25.  Upper  air  error  statistics,  November  -  Korea.  (Continued) 
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Upper  Air 

The  main  feature  in  the  500  mb  geopotential  field  was  a  closed  trough  over  the  southern  Turkey  coast. 

All  models  handled  this  similarly  although  with  slightly  different  central  heights.  RWM  had  the  most 
diffuse  centered  and  located  further  east  than  the  others  did.  GSM  had  the  lowest  central  height  of  5520 
meters,  RAMS  the  highest  at  5580  meters.  All  models  placed  the  500  mb  jet  maximum  over  the 
northern  tip  of  the  Red  Sea,  except  for  RWM  which  moved  it  eastward  over  northwest  Saudi  Arabia. 

Figures  4-26(a  -  e)  depict  the  geopotential  height. 

Phenomenology  results  at  24  hours  into  the  second  forecast  period  (17  Nov.  94  -  0000  UTC). 

Surface  / 

Thirty-six  hours  later,  the  low  from  the  eastern  Mediterranean  was  moved  by  the  models  eastward. 

RAMS,  MM5,  and  GSM  had  the  center  located  over  northwest  Iran,  while  NORAPS6  lagged  the  center 
farther  behind  in  northeastern  Iraq.  RWM  again  had  an  extremely  noisy  sea  level  pressure  field. 

RAMS,  MM5,  and  NORAPS6  developed  a  stronger  ridging  over  Egypt  and  Saudi  Arabia  than  GSM. 


geopotential  height  (m) 


(a)  GSM  Analysis  at  15  Nov  94-  1200  UTC 


Figure  4-26.  Geopotential  height  -  Middle  East. 
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(b)  MM5  -  15  Nov  94-  1200  UTC  forecast 


(c)  NORAPS6  -  15  Nov  94-  1200  UTC 

Figure  4-26.  Geopotential  height  -  Middle  East.  (Continued) 
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(d)  RAMS  -  15  Nov  94  -  1200  UTC  forecast 


(e)  RWM  -  15  Nov  94  -  1200  UTC  forecast 
Figure  4-26.  Geopotential  height  -  Middle  East.  (Continued) 
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Upper  Air 

At  500  mb,  all  regional  models  developed  a  double-core  structure  to  the  mid-level  jet  stream  while  GSM 
maintained  a  single  core  over  Kuwait.  RWM  had  the  weakest  core  maximum  of  3 1  m/s,  NORAPS6  38 
m/s,  RAMS  37  m/s,  and  MM5  40  m/s. 

4. 1.3. 9  Statistical  Results  -  Middle  East.  Error  frequency  distributions  were  generated  for  temperature, 
surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20  knots,  and 
wind  direction.  Data  from  all  forecast  times,  both  November  forecasts,  and  where  appropriate,  levels 
were  merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets  of  figures  were 
developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large  number  of  surface 
observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air  observations. 
Figures  4-27(a  -  f)  are  the  surface  results  and  Figures  4-28(a  -  e)  are  the  upper  air  results. 
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(a)  Temperature 

Figure  4-27.  Surface  error  statistics,  November  -  Middle  East. 
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Figure  4-27.  Surface  error  statistics,  November  -  Middle  East.  (Continued) 
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Figure  4-27.  Surface  error  statistics,  November  -  Middle  East.  (Continued) 
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(d)  Wind  speed  <  20  Knots 

Figure  4-27.  Surface  error  statistics,  November  -  Middle  East.  (Continued) 
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Figure  4-28.  Upper  air  error  statistics,  November  -  Middle  East.  (Continued) 
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Figure  4-28.  Upper  air  error  statistics,  November  -  Middle  East.  (Continued) 
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Figure  4-28.  Upper  air  error  statistics,  November  -  Middle  East.  (Continued) 
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Figure  4-28.  Upper  air  error  statistics,  November  -  Middle  East.  (Continued) 
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4.1.3.10  Phenomenological  Results  -  Central  America.  Phenomenology  results  at  24  hours  into  the  first 
forecast  period  (15  Nov.  94  -  1200  UTC). 

Surface 

The  most  significant  surface  feature  in  the  region  at  this  time  was  a  surface  cyclone  located  near  the 
southern  tip  of  Florida.  All  models  had  the  center  located  in  a  similar  position  with  GSM  having  a  1006 
mb  center,  MM5  1004  mb,  RAMS  1009  mb,  NORAPS 6  1008  mb,  and  RWM  1009  mb  and  again 
extremely  noisy.  RWM  did  not  have  a  well-defined  circulation  around  the  center,  while  all  other  models 
showed  a  closed  circulation  of  up  to  12  m/s  maximum  winds.  Both  the  pressure  and  wind  fields  are 
presented  in  Figures  4-29(a  -  e)  and  Figures  4-30(a  -  e). 

Upper  Air 

The  cyclonic  circulation  continued  up  into  the  upper  atmosphere.  The  maximum  500  mb  winds  were 
20  m/s  for  GSM,  RWM,  and  RAMS  and  19  m/s  for  NORAPS6.  MM5  generated  localized  regions  of 
27  m/s  due  to  momentum  transfer  in  its  convective  parameterization. 


(a)  GSM  Analysis  at  15  Nov  94-  1200  UTC 


Figure  4-29.  Surface  pressure  -  Central  America. 
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(d)  RAMS  -  15  Nov  94-  1200  UTC  forecast 


(e)  RWM  -  15  Nov  94  -  1200  UTC  forecast 
Figure  4-29.  Surface  pressure  -  Central  America.  (Continued) 


ISO 


speed  (m/s) 

(a)  GSM  Analysis  at  15  Nov  94-  1200  UTC 


(b)  MM5  -  15  Nov  94  -  1200  UTC  forecast 

Figure  4-30.  Surface  winds  -  Central  America.  (Continued) 
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(d)  RAMS  -  15  Nov  94  -  1200  UTC  forecast 
Figure  4-30.  Surface  winds  -  Central  America.  (Continued) 
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(e)  RWM  -  15  Nov  94  -  1200  UTC  forecast 


Figure  4-30.  Surface  winds  -  Central  America.  (Continued) 

Phenomenology  results  at  24  hours  into  the  second  forecast  period  (17  Nov.  94  -  0000  UTC). 

Surface 

Thirty-six  hours  later,  the  cyclone  had  moved  to  the  north  boundary  of  the  domain  where  the  models 
could  not  handle  it  properly.  RAMS,  MM5,  and  GSM  started  to  develop  another  cyclonic  circulation, 
just  off  the  southern  coast  of  El  Salvador.  RWM  and  NORAPS6  did  not  form  this  low-pressure 
circulation. 


Upper  Air 

This  developing  cyclone  was  also  reflected  in  the  500  mb  flow  field.  RAMS  had  up  to  a  12  m/s 
maximum  speed  around  the  500  mb  center,  MM5  about  1 1  m/s,  and  GSM  about  10  m/s.  NORAPS6  and 
RWM  had  -vinds  from  the  east  to  southeast  at  5-8  m/s. 

4. 1 .3. 1 1  Statistical  Results  -  Central  America.  Error  frequency  distributions  were  generated  for 
temperature,  surface  pressure,  dew  point  depression,  wind  speed  below  20  knots,  wind  speed  above  20 
knots,  and  wind  direction.  Data  from  all  forecast  times,  both  November  forecasts,  and  where 
appropriate,  levels  were  merged.  The  central  column  corresponds  to  the  desired  error  bounds.  Two  sets 
of  figures  were  developed  to  describe  the  forecast  results.  One  is  a  surface  set  that  includes  the  large 
number  of  surface  observations  and  the  other  is  an  upper  air  set  that  includes  the  more  sparse  upper  air 
observations.  Figures  4-3 1  (a  -  f)  are  the  surface  results  and  Figures  4-32(a  -  e)  are  the  upper  air  results. 
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Figure  4-3 1 .  Surface  error  statistics,  November  -Central  America. 
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Figure  4-31.  Surface  error  statistics,  November  -Central  America.  (Continued) 
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Figure  4-31.  Surface  error  statistics,  November  -Central  America.  (Continued) 
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Figure  4-31.  Surface  error  statistics,  November -Central  America.  (Continued) 
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Figure  4-32.  Upper  air  error  statistics,  November  -  Central  America. 
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Figure  4-32.  Upper  air  error  statistics,  November  -  Central  America.  (Continued) 
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Figure  4-32.  Upper  air  error  statistics,  November  -  Central  America.  (Continued) 
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Figure  4-32.  Upper  air  error  statistics,  November  -  Central  America.  (Continued) 
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Figure  4-32.  Upper  air  error  statistics 
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4.2  DATA  DENIED  IN  KOREAN  THEATER. 


4.2.1  Objective. 

This  test  explored  each  model’s  ability  to  provide  a  valid  forecast  when  normally  available  observations 
cannot  be  obtained.  Two  possibilities  will  be  considered.  For  data  denied  by  enemy  action,  we  will 
eliminate  all  surface  and  upper  air  data  from  North  Korea  and  China.  The  second  case  will  consider  a 
communications  failure  and  a  single  observation  point.  Upper  air  observation  in  Seoul  will  be  used  to 
initialize  the  forecast.  These  will  be  compared  for  all  models  with  forecasts  made  with  all  the 
observation  data  and  NMC  data. 

4.2.2  Description. 

The  72  hour  period  from  1200Z,  27  Sep  93  to  1200Z,  30  Sep  93  was  chosen  due  to  the  presence  of  an 
impressive  cutoff  low  at  500  mb  to  the  north  of  the  Korean  peninsula.  This  cutoff  low  traversed  almost 
due  east  across  northeast  China  from  the  eastern  border  of  China  and  Mongolia  to  the  coast,  and  was 
reinforced  by  another  shortwave  moving  in  the  jet  to  the  north  at  the  end  of  the  period.  The  upper  level 
feature  forced  at  least  two  cold  fronts  through  the  Korean  peninsula;  the  first  around  0000Z,  29 
September  and  a  second  stronger  front  around  0000Z,  30  September. 

4.2.3  Data  Source. 

NMC  gridded  data  are  contained  within  the  NCAR  data  archives  in  ds082.0,  tape  Y17440.  The 
rawinsonde  data  are  in  ds353.1,  tape  Y17946  while  the  surface  observations  at  0  UTC  and  12  UTC  are 
in  ds464.0,  tape  Y 17936. 

4.2.4  Phenomenology  Results  -  All  Data. 

In  this  calculation,  all  available  surface  and  upper  air  data  in  the  theater  was  used.  It  provides  a 
comparison  for  the  remainder  of  the  denied  cases. 

Surface 

The  surface  meteorology  was  dominated  by  a  low-pressure  system  with  a  double  center  analyzed  at  1004 
mb  to  NNW  of  Korea  and  1008  mb  NE  of  Korea.  RAMS  predicted  a  single  center  of  1006  mb  although 
it  produced  a  lobe  toward  the  region  of  the  second  center.  MM5  also  produced  a  single  elongated  center 
more  closely  coinciding  with  the  analyzed  center  to  the  NE  of  Korea.  NORAPS6  and  MM5  did  produce 
a  dual  center  structure  although  RWM  probably  under-predicted  the  secondary  center. 

Upper  Air 

The  500  mb  level  geopotential  field  was  dominated  by  the  upper  air  support  for  the  surface  low.  A 
strong  closed-core  circulation  with  an  analyzed  central  height  of  5310  m  was  located  to  the  NNW  of 
Korea.  All  models  had  a  good  location,  with  RAMS  having  a  central  height  of  5400  m,  MM5  5340  m, 
NORAPS6  5340  m,  and  RWM  5370  m. 

4.2.5  Phenomenology  Results  -  No  data  from  China  and  North  Korea. 

For  the  data-denied  run,  the  models'  results  were  not  very  different  from  the  full  data  run.  All  models 
showed  similar  behavior;  the  surface  low  was  just  slightly  weaker  as  was  the  upper  air  circulation.  This 
is  an  understandable  behavior  since  there  is  more  of  a  reliance  on  the  smoother  gridded  pressure  data  in 
generating  the  initial  and  boundary  conditions  rather  than  using  the  additional  observational  data.  Also, 
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because  of  the  relatively  coarse  grid  resolution  used  in  these  runs  and  the  omission  of  even  a  large 
number  of  observations,  as  long  as  the  gridded  data  is  available,  it  is  not  as  detrimental  to  the  forecast 
results  as  compared  to  a  run  with  higher  grid  resolutions.  Figures  4-33(a  -  h)  shows  the  surface  pressure 
maps  with  all  data  and  the  same  maps  for  the  no  data  from  North  Korea  and  China.  Corresponding  maps 
of  the  geopotential  height  are  shown  on  Figures  4-34(a  -  h).  All  data  are  for  the  forecast  fields  at  1200 
UTC  on  28  September  1993. 


(a)  MM5- all  data 

Figure  4-33.  Surface  pressure  -  Korea. 
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(d)  N0RAPS6  -  no  data  from  North  Korea  and  China. 


(e)  RAMS  -  all  data 

Figure  4-33.  Surface  pressure  -  Korea.  (Continued) 
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(g)  RWM  -  all  data 

Figure  4-33.  Surface  pressure  -  Korea.  (Continued) 
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(h)  RWM  -  no  data  from  North  Korea  and  China. 
Figure  4-33.  Surface  pressure  -  Korea.  (Continued) 
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(c)  NORAPS 6  -  all  data 


(d)  NORAPS6  -  no  data  from  North  Korea  and  China. 
Figure  4-34.  Geopotential  height  -  Korea.  (Continued) 
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(e)  RAMS  -  all  data 


Figure  4-34.  Geopotential  height  -  Korea.  (Continued) 


(g)  RWM  -  all  data 


(h)  RWM  -  no  data  from  North  Korea  and  China 

Figure  4-34.  Geopotential  height  -  Korea.  (Continued) 
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4.2.6  Phenomenology  Results  -  Single  Sounding  Run. 

As  expected,  the  run  with  the  single  Seoul  sounding  showed,  in  a  synoptically  disturbed  environment 
such  as  this,  that  a  NWP  model  will  be  unable  to  produce  a  reasonable  forecast  if  initialized  from  a 
limited  observation  set.  If  the  meteorological  situations  were  less  disturbed  and  the  main  flows  were 
physiographically  forced,  the  NWP  model  would  have  a  better  chance. 

RAMS  was  the  only  model  that  had  a  standard  option  to  make  a  run  with  a  single  sounding.  For  the 
remaining  models,  we  generated  a  horizontally  homogeneous  pressure  data  file  from  the  Seoul  sounding 
and  allowed  their  regular  data  analysis  procedure  to  process  the  data.  However,  this  produces  a  large 
imbalance  with  the  initial  winds  and  pressure  fields  in  the  three  models.  Because  RAMS  has  an  option 
for  this  type  of  run,  it  has  a  feature  that  assumes  an  initial  geostrophic  balance  between  the  winds  and 
pressure.  The  other  models  spent  the  majority  of  the  run  attempting  to  adjust  to  the  initial  imbalance. 

4.3  OBJECTIVE  OR  ENHANCED  TESTS. 


4.3.1  Objective. 

Requirements  for  the  TFM  specify  an  objective  resolution  of  6  nmi.  Forecast  models  are  sensitive  to  the 
resolution  used  and  should  produce  a  “better”  forecast  at  finer  scales.  These  tests  demonstrate  each 
model’s  sensitivity  to  scale  and  indicate  whether  the  forecasts  will  improve  at  a  finer  scale. 

4.3.2  Description. 

The  objective  model  resolution  of  6  nmi  was  investigated  with  MM5,  RAMS,  and  NORAPS6  using  the 
nested  grid  configurations  of  these  models.  For  the  Central  America  (focusing  on  Panama)  and  the 
Middle  East  (focusing  on  Kuwait/Iraq)  regions,  we  configured  these  models  in  a  2-grid  configuration 
with  the  finer  grid  spacing  at  approximately  the  6  nmi  objective.  RAMS  allows  for  a  user-selectable 
spacing  ratio  between  the  grids,  so  it  would  be  possible  to  take  the  coarse  grid  as  the  25  nmi  grid  used  in 
the  threshold  tests  and  use  a  4-1  grid  space  ratio.  However,  both  MM5  and  NORAPS6  allowed  only  a  3- 
1  grid  space  ratio.  Therefore,  we  reconfigured  a  coarse  grid  domain  with  an  18  nmi  spacing  and  ran  all 
models  with  a  nested  grid  at  a  3-1  grid  space  ratio.  Because  of  the  finer  spacing,  we  will  use 
approximately  98x98  grid  points  on  the  coarse  grid  to  cover  the  1500  x  1500  nmi  domain.  By  covering 
the  same  domain  as  the  threshold  runs,  we  provided  for  an  additional  sensitivity  comparison  for  these 
models  to  test  a  28  percent  reduction  in  grid  spacing.  Nested  grids  were  ran  with  71x71  grid  points. 

4.3.3  Data  Source. 

This  remains  the  same  as  the  first  period  of  August  94  Threshold  Tests. 

The  main  differences  in  a  model  ran  between  25  and  6  nmi  grid  spacing  was  the  ability  to  resolve  the 
physiographic  forcing  (topography,  land-use,  coastline,  etc.).  The  August  period  was  expected  to  be  the 
best  period  for  sea  breeze  formation,  thus  testing  several  aspects  of  the  models’  physics  (radiation,  land- 
use,  etc.)  directly  without  the  potentially  stronger  synoptic  forcing  of  the  November  1994  period. 

4.3.4  Phenomenology  Results. 

The  objective  runs  used  an  18  nmi  coarse  resolution  domain  over  the  same  area  as  the  threshold  ran  for 
the  Central  American  and  Middle  Eastern  domains  for  the  ran  beginning  on  16  August  94  0000  UTC 
run.  A  nested  grid  of  6  nmi  spacing  was  placed  over  Panama  and  the  Persian  Gulf  regions.  NORAPS6 
and  RAMS  were  run  in  this  manner,  but  we  were  unable  to  successfully  start  a  ran  with  MM5,  even  after 


205 


consultation  with  the  developers  at  NCAR.  Therefore,  the  following  discussion  will  only  pertain  to 
RAMS  and  NORAPS6. 


4.3.4. 1  Middle  East.  The  surface  flow  field  over  the  Persian  Gulf  was  simulated  quite  differently  with 
RAMS  with  the  higher  resolution  grid.  While  the  flow  in  the  objective  run  was  about  3  m/s  from  the 
NE,  RAMS  produced  2  m/s  flow  from  the  SE  along  the  SW  Gulf  coast  NORAPS6  did  not  show  this 
variability,  producing  NW  winds  in  both  runs.  Figure  4-35  presents  the  grid  relationship.  Figure  4-36a 
is  a  map  of  the  NORAPS6  predicted  winds  in  the  25  nmi  threshold  calculation  and  Figure  4~36b  is  in  the 
6  nmi  nested  grid.  Figures  4-36(c  and  d)  present  the  same  data  for  RAMS. 


Figure  4-35.  Middle  East  -  full  and  nested  grids. 


(b)  N0RAPS6  -  6  nmi  grid  spacing 
Figure  4-36.  Surface  Winds  -  Middle  East. 
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(c)  RAMS  -  25  nmi  grid  spacing 


(d)  RAMS  -  6  nmi  grid  spacing 
Figure  4-36.  Surface  Winds  -  M’  '  -  Fast.  (Continued) 
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4.3.4.2  Central  America.  The  higher  resolution  of  the  nested  grid  did  a  much  better  job  of  capturing  a 
sea  breeze  circulation  than  the  previous  coarser  resolution  runs.  While  NORAPS6  in  the  earlier  runs  did 
not  have  a  sea  breeze  convergence  zone  along  the  southern  Panama  coast,  the  objective  run  produced  a 
well-defined  zone.  RAMS  also  produced  a  convergence  zone  with  slighdy  more  local  variability  than 
NORAPS6.  The  coarse  grid  fields  except  in  the  area  of  the  nested  grid  were  not  significantly  different 
between  the  18  nmi  used  here  and  25  nmi  runs  used  for  the  threshold  tests.  Figure  4-37  presents  the  grid 
relationship.  Figure  4-38a  is  a  map  of  the  NORAPS6  predicted  winds  in  the  25  nmi  threshold 
calculation  and  Figure  4-3  8b  in  the  6  nmi  nested  grid.  Figures  4-3  8(c  and  d)  present  the  same  data  for 
RAMS. 


Figure  4-37.  Central  America  -  full  and  nested  grids. 
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(a)  N0RAPS6  -  25  nmi  grid  spacing 


(b)  N0RAPS6  -  6  nmi  grid  spacing 
Figure  4-38.  Surface  Winds  -  Central  America. 
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(d)  RAMS  -  6  nmi  grid  spacing 
Figure  4-38.  Surface  Winds  -  Central  America.  (Continued). 
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SECTION  5 

MODEL  COMPARISON  SUMMARY 


The  previous  sections  presented  voluminous  model  results  produced  to  make  detailed  comparisons  of 
the  four  models.  However,  the  shear  volume  of  the  data  makes  drawing  conclusions  difficult.  To 
facilitate  these  comparisons  several  generalized  procedures  are  developed.  In  this  section  we  develop 
and  provide  this  summary  model  comparison  data. 

5.1  RANK  SCORING. 

We  have  carefully  reviewed  the  entire  set  of  statistics.  From  casual  inspection  of  those  statistics  it  is 
difficult  to  identify  a  clear  trend.  However,  we  have  applied  a  simple  rank  scoring  methodology  that 
does  identify  a  clear  result  from  the  morass  of  detailed  statistics.  This  rank  scoring  is  applied  to  the 
threshold  cases  and  not  to  the  data  denied  or  objective  excursions.  Merging  the  excursions  into  the  other 
data  would  add  an  unnecessary  level  of  complexity. 

Rank  scoring  is  a  particularly  convenient  way  for  assessing  the  relative  performance  of  each  model  over 
the  diverse  set  of  quantities  that  each  model  produces.  For  example,  we  can  compare  each  model’s 
predictions  of  surface  pressures,  temperatures,  dew  points,  and  wind  speeds  and  directions  with  available 
observations  and  obtain  quantitative  error  measures  for  each  of  those  quantities  individually.  However, 
we  also  need  to  combine  the  error  measures  for  these  physically  distinct  quantities  into  a  single  overall 
measure  of  model  performance.  It  is  not  a  priori  obvious,  for  example,  how  to  combine  an  error  in 
temperature  with  an  error  in  wind  direction. 

Because  one  of  our  primary  concerns  is  determining  the  overall  performance  of  each  model  relative  to 
all  of  the  other  models,  rank  scoring  provides  a  useful  and  powerful  tool  for  combining  all  the  diverse 
error  measures  into  a  single  performance  measure  for  each  model.  We  apply  rank  scoring  by  computing 
each  model’s  error  for  each  available  observation.  For  each  observation  we  rank  all  four  models:  the 
model  with  the  smallest  error  is  ranked  1,  that  with  the  next  lowest  error  is  ranked  2,  etc.  Then  we 
examine  all  these  accumulated  rankings  and,  for  each  model,  we  determine  how  many  times  it  ranked 
number  1,  how  many  times  it  ranked  number  2,  etc. 

For  this  model  comparison,  we  considered  only  the  extreme  rankings:  how  many  times  each  model  was 
the  best  (i.e.  ranked  number  1)  and  how  many  times  it  was  the  worst  (i.e.  ranked  number  4).  Focusing 
on  the  extremes  will  help  identify  any  model  that  is  particularly  outstanding  or  particularly  poor.  If  all 
four  models  were  of  similar  quality,  we  would  expect  the  one  that  was  best  and  the  one  that  was  worst 
for  any  individual  observation  would  be  largely  a  matter  of  chance.  Therefore,  each  model  would  be 
“Best”  about  25  percent  of  the  time  and  “Worst”  about  25  percent  of  the  time.  If  a  model  is  “Best” 
significantly  more  than  25  percent  of  the  time  or  “Worst”  significantly  less  than  25  percent  of  the  time, 
then  that  model  would  be  judged  to  be  significantly  better  than  the  others.  Note  that  rank  scoring  is  like 
a  zero-sum  game — if  one  model  is  “Best”  significantly  more  than  25  percent  of  the  time  then,  on 
average,  the  other  three  models  must  be  “Best”  significantly  less  than  25  percent  of  the  time. 

We  applied  rank  scoring  by  determining  which  one  had  the  smallest  RMS  error  for  each  observation. 
This  approach  rates  each  model’s  general  performance  without  regard  to  the  accuracy  requirements  for 
the  various  quantities.  Our  scoring  system  uses  three  of  the  statistics  that  we  generated  for  each  time, 
variable,  and  level: 

1 .  Temperature  RMSE 
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2.  Relative  humidity  RMSE 

3.  Wind  RMSVE 

For  each  case  we  ran,  we  rank-ordered  the  four  models  from  which  one  did  best  to  which  one  did  worst 
for  each  of  these  three  statistics.  Then  we  tabulated  for  each  model  the  total  number  of  times  it  gave  the 
best  result  and  the  total  number  of  times  it  gave  the  worst  result.  We  combined  results  for  verification 
times  of  0, 12,  24,  and  36  hours  and  two  36  hour  runs  for  each  season.  This  data  is  summarized  in  Table 
5-1. 

Table  5-1.  RMS  rank  ordering. 


Theater 

Scores 

MM5 

NORAPS6 

RAMS 

RWM 

Total 

Best  Worst 

Best 

Worst 

Best 

Worst 

Best 

Worst 

Continental 
United  States 

74 

21 

19 

86 

117 

7 

22 

118 

232 

Alaska 

65 

52 

33 

51 

143 

15 

23 

146 

264 

Korea 

36 

135 

26 

57 

177 

5 

25 

67 

264 

Middle  East 

22 

187 

34 

27 

168 

9 

40 

41 

264 

Central  America 

29 

93 

27 

93 

169 

5 

39 

73 

264 

Alaska 

55 

38 

44 

34 

145 

19 

20 

173 

264 

Korea 

73 

45 

49 

71 

122 

26 

20 

122 

264 

Middle  East 

54 

46 

56 

39 

113 

25 

23 

136 

246 

Central  America 

72 

40 

34 

45 

95 

37 

32 

111 

233 

Total 

490 

657 

322 

503 

1249 

148 

244 

987 

2295 

5.2  SATISFYING  AWS  CRITERIA. 

The  previous  comparison  provides  a  means  of  comparing  the  model’s  performance  but  does  not  provide 
a  measure  of  how  well  the  predictions  agreed  with  the  observed  data.  This  is  accomplished  in  this 
section  by  comparing  the  forecast  data  to  the  observations  and  determining  the  number  of  times  the 
predicted  value  was  within  the  error  criteria  of  the  observed  value.  The  previously  developed  weather 
data  accuracy  criteria  are  repeated  in  Table  5-2.  These  criteria  were  used  as  a  measure  of  each  model’s 
forecast  skill.  Initially,  all  quantities  were  grouped  together  for  each  case  and  region  to  provide  a  net 
total  of  how  often  each  model  satisfies  criteria  as  was  done  in  the  previous  comparison. 

Table  5-2.  Data  accuracy. 


Parameter 


Temperature 


Dewpoint  Depression 


Criteria 


Wind  Speed 

less  than  10  meters/sec 

1  meter/sec 

greater  than  10  meters/sec 

2.5  meters/sec 

Wind  Direction 

30  degrees 

Sea  Level  Pressure 

1.7  millibars 
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5.2.1  Performance  by  Theater. 


Table  5-3  expresses  each  model’s  performance  as  the  percentage  of  times  that  its  predicted  value  is 
within  the  error  bounds  specified. 


Table  5-3.  Predictions  within  criteria  (see  Table  5-2)  by  theater. 


Model 

MM5 

NORAPS6 

RAMS 

RWM 

Theater 

Percent 

Percent 

Percent 

Percent 

Observations 

1 

Continental  United  States 

61 

52 

64 

48 

24971 

2 

Alaska 

60 

53 

64 

46 

4855 

2 

Central  America 

46 

44 

57 

48 

6231 

2 

Korea 

45 

44 

56 

46 

31427 

2 

Middle  East 

34 

40 

49 

40 

11392 

3 

Alaska 

52 

45 

59 

38 

6056 

3 

Central  America 

48 

38 

58 

40 

4620 

3 

Korea 

46 

38 

46 

38 

20961 

3 

Middle  East 

43 

39 

48 

36 

7114 

Total 

49 

44 

55 

43 

117627 

RAMS  consistently  produced  better  results  for  the  criteria  in  areas  other  than  the  Continental  United 
States.  There  are  potentially  several  reasons  for  these  differences.  One  reason  could  have  been  the  skill 
of  the  analysis  routine  used  to  develop  the  initial  gridded  fields  from  the  observations.  The  next  analysis 
was  accomplished  to  test  this  hypothesis. 

5.2.2  Performance  by  Forecast  Hour. 

In  this  section,  the  relative  performance  by  forecast  hour  is  examined.  The  stratification  was  done  only 
by  hour,  not  by  region/hour.  The  relatively  large  number  of  surface  observations,  compared  to  upper  air 
observations  was  responsible  for  developing  two  tables.  Table  5-4  presents  the  surface  comparison  and 
Table  5-5  the  upper  air  comparison  for  each  forecast  period. 

Table  5-4.  Surface  predictions  within  criteria  (see  Table  5-2)  by  forecast  time. 


MM5 

NORAPS6 

RAMS 

RWM 

Parameter 

Forecast 

Time 

% 

% 

% 

% 

Total 

Observations 

Temperature 

0  hour 

65 

38 

66 

30 

3800 

Dewpoint  Depression 

0  hour 

35 

9 

59 

21 

3733 

Wind  Speed  <10 

0  hour 

73 

27 

67 

47 

3735 

Wind  Speed  >10 

0  hour 

88 

60 

100 

84 

129 

Wind  Direction 

0  hour 

54 

35 

53 

39 

3864 

Pressure 

0  hour 

63 

65 

46 

44 

2080 

Temperature 

12  hour 

32 

35 

42 

29 

3784 
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Table  5-4.  Surface  predictions  within  criteria  (see  Table  5-2)  by  forecast  time  (Continued). 


MM5 

NORAPS6 

RAMS 

RWM 

Parameter 

Forecast 

Time 

% 

% 

% 

% 

Total 

Observations 

Dewpoint  Depression 

12  hour 

40 

29 

39 

32 

3722 

WindS 

peed  <10 

12  hour 

24 

28 

41 

29 

3715 

WindS 

peed  >10 

12  hour 

81 

89 

96 

76 

139 

Wind  Direction 

12  hour 

38 

38 

40 

36 

3854 

Pressur 

a 

12  hour 

47 

38 

42 

41 

2037 

Temperature 

24hour 

31 

38 

44 

34 

3789 

Dewpoint  Depression 

24  hour 

44 

33 

41 

38 

3724 

WindS 

SlifiSISBH 

24  hour 

22 

28 

39 

28 

3696 

WindS 

24  hour 

82 

91 

97 

74 

146 

Wind  Direction 

24  hour 

36 

36 

39 

34 

3842 

Pressure 

24  hour 

46 

30 

34 

39 

2078 

Temperature 

36  hour 

29 

38 

42 

31 

3745 

Dewpoint  Depression 

36  hour 

43 

37 

45 

38 

3674 

WindS 

peed  <  10 

36  hour 

26 

30 

37 

28 

3692 

WindS 

peed  >10 

36  hour 

90 

97 

97 

83 

115 

Wind  Direction 

36  hour 

33 

34 

35 

33 

3807 

Pressure 

36  hour 

34 

24 

34 

34 

2062 

For  the  surface  parameters  at  the  analysis  time  (0  hour),  two  distinct  groups  appear  with  MM5  and 
RAMS  with  the  superior  scores  and  the  other  two  models  in  the  second  group.  (The  comparison  at  the 
zero  hour  provides  a  score  for  the  initial  fields  prepared  by  the  model’s  analysis  routine.  The  same  data 
was  supplied  to  each  model.)  However,  by  12  hours,  a  significant  portion  of  the  MM5  and  RAMS 
advantage  had  disappeared.  These  two  models  were  still  the  top  performers  with  RAMS  performing 
better  than  MM5  each  time  for  most  variables. 

Only  in  predicting  sea  level  pressure  did  MM5  regularly  outperform  RAMS.  NORAPS6  and  RWM 
performed  about  the  same  throughout  the  forecast  period  except  for  dewpoint  depression.  NORAPS6 
and  RWM’s  predictions  of  dewpoint  depression  were  very  poor  initially,  but  become  competitive  at  later 
times.  The  poor  dewpoint  analysis  by  NORAPS6  is  easily  explained.  NORAPS6  used  the  humidity  data 
from  the  global  gridded  data  and  did  not  use  the  relative  humidity  data  from  the  observations  to  generate 
the  initial  fields.  Each  model’s  skill  at  12,  24,  and  36  hours  was  relatively  stable  with  a  small 
degradation  with  time.  The  performance  scores  (Table  5-4)  are  lower  for  the  surface  comparisons  than 
the  corresponding  upper  air  comparison  (Table  5-5).  These  differences  can  probably  be  attributed  to  two 
factors.  First,  even  though  several  of  the  models  have  “surface”  output  options  there  will  be  differences 
between  the  actual  observation  height  and  the  computed  height.  There  was  no  standard  method  used. 

For  instance,  RAMS  used  the  lower  cell  parameters  without  correction,  except  for  sea  level  pressure. 
This  cell  center  is  at  46  meters  and  velocities  at  this  point  should  have  a  small  positive  bias  as  the 
observations  are  usually  taken  at  10  meters.  The  remaining  models  varied  in  how  each  computed  the 
surface  data.  These  output  differences  were  probably  obscured  by  the  sub-grid  variability  present  in  the 
real  world.  Local  structures,  terrain  features,  trees,  etc.  affected  the  observed  data.  With  a  grid  size  in 
excess  of  40  km,  none  of  these  local  features  could  be  modeled. 
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Table  5-5.  Upper  air  predictions  within  criteria  (see  Table  5-2)  by  forecast  time. 


MM  5 

NORAPS6 

RAMS 

RWM 

Parameter 

Forecast 

% 

% 

% 

% 

Total 

Time 

Observations 

Temperature 

0  hour 

88 

85 

93 

62 

3295 

Dewpoint  Depression 

0  hour 

28 

9 

50 

19 

2455 

Wind  Speed  <10 

0  hour 

80 

60 

82 

52 

1119 

Wind  Speed  >10 

0  hour 

90 

76 

93 

71 

2000 

Wind  Direction 

0  hour 

90 

83 

94 

70 

3119 

Temperature 

12  hour 

64 

70 

74 

65 

3356 

Dewpoint  Depression 

12  hour 

24 

14 

29 

13 

2509 

Wind  Speed  <10 

12  hour 

48 

54 

61 

53 

1137 

Wind  Speed  >10 

12  hour 

74 

73 

82 

74 

1975 

Wind  Direction 

12  hour 

72 

75 

80 

73 

3112 

Temperature 

24  hour 

57 

67 

70 

65 

3457 

Dewpoint  Depression 

24  hour 

24 

16 

26 

12 

2599 

Wind  Speed  <10 

24  hour 

50 

50 

63 

53 

1239 

Wind  Speed  >10 

24  hour 

73 

71 

81 

74 

2045 

Temperature 

36  hour 

54 

63 

68 

63 

3287 

Dewpoint  Depression 

36  hour 

22 

16 

24 

10 

2465 

Wind  Speed  <10 

36  hour 

54 

54 

64 

55 

1259 

Wind  Speed  >10 

36  hour 

70 

72 

80 

75 

1847 

Wind  Direction 

36  hour 

63 

73 

76 

75 

3106 

A  similar  forecast  accuracy  time  response  can  be  seen  in  the  upper  air  comparison,  however,  all  the 
scores  are  higher.  Sub-grid  influences  are  much  less  of  a  factor  above  the  surface.  For  each  parameter 
and  forecast  time,  RAMS  produced  the  predictions  that  most  often  met  the  criteria.  NORAPS6  often 
produced  more  accurate  predictions  than  MM5  in  the  upper  air  calculations.  However,  there  were  cases 
when  every  model  had  difficulty  accurately  predicting  a  parameter;  specifically,  the  predictions  of 
dewpoint  depression,  where  the  highest  forecast  accuracy  seen  was  29  percent.  The  initialization  of  the 
dewpoint  depression  varied  widely  in  accuracy  and  this  may  be  reflected  in  these  scores.  This  lack  of 
consistency  in  upper  level  moisture  observations  is  most  likely  the  reason  for  this  poor  performance. 
(Dewpoint  was  only  validated  at  points  where  relative  humidity  data  existed.  The  lack  of  this  data  at 
some  locations  above  300  mb  is  reflected  in  the  reduced  number  of  observations.) 

The  statistical  data,  when  depicted  by  forecast  time  (Tables  5-4  and  5-5)  indicates  that  RAMS  and  MM5 
have  better  analysis  routines.  However,  the  statistical  advantage  of  the  better  analysis  is  not  obvious  by 
12  hours  into  the  forecast  period.  The  scores  at  12  through  36  hours  were  relatively  consistent.  The 
ranking  of  the  models  remained  the  same.  Statistically,  there  appears  to  be  no  compelling  reason  to  use 
a  better  analysis  routine  to  initialize  the  forecasts.  However,  the  largely  non-linear  atmospheric 
processes  modeled  by  these  models  makes  this  statement  somewhat  risky.  During  a  software  upgrade 
for  NORAPS6,  a  minimal  impact  in  forecast  accuracy  was  noted  as  a  result  of  improving  the  analysis 
(Hodur,  1987).  This  somewhat  corroborates  the  statistical  results  and  is  probably  the  reason  NORAPS6 
produces  such  balanced  accuracy.  There  are  more  technically  advanced  analysis  routines  that  all  models 
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could  use  to  improve  the  initial  accuracy  and  potentially  the  overall  results.  The  above  data  suggests  the 
actual  forecast  model  is  determining  the  accuracy  score  after  some  start  up  period. 

5.3  EXECUTION  TIME  AND  HARDWARE  SELECTION. 

In  evaluating  the  four  candidate  forecast  models  we  emphasized  each  model’s  forecast  performance  and 
accuracy  because  good  performance  is  generally  more  difficult  and  costly  to  achieve  than  is  reduced  run 
time.  However,  run  time  is  important  in  an  operational  environment,  and  the  AF  requirement  is  to 
produce  a  72  hour  forecast  in  1  hour.  All  tested  models  ran  substantially  slower  than  this  on  an  IBM 
RS/6000  Model  370  workstation.  Average  run  times  for  36  hour  forecasts  were  RWM,  2.8  hr; 
NORAPS6, 4.3  hr;  RAMS,  8.8  hr,  and  MM5,  ~  18  hr.  We  can  meet  the  AFs  run  time  requirement  by 
speeding  up  model  execution,  by  using  faster  computers,  or  by  a  combination  of  both. 

5.3.1  Code  Improvements. 

We  ran  each  of  the  candidate  forecast  models  using  the  best  physics  models  each  one  offered.  It  is 
unrealistic  to  relax  this  choice  because,  even  using  their  best  physics,  none  of  the  models  fully  met  the 
AF’s  forecast  performance  criteria.  In  fact,  the  converse  would  appear  to  be  true:  additional  and  better 
physics  models  must  be  added  to  improve  forecast  ability  and  to  predict  phenomena,  such  as  clouds,  that 
are  not  provided  by  current  models. 

In  general  it  is  more  cost-effective  to  speed  up  existing  physics  models  than  to  develop  new  models.  We 
can  improve  the  performance  of  existing  detailed  physical  models  by  parameterizing  their  output  or  by 
approximating  parts  of  them  with  analytic  functions.  Of  the  four  TFM  candidates,  RAMS  and  MM5 
could  benefit  most  from  this  type  of  improvement  because  they  already  contain  the  most  sophisticated 
physics  models.  It  is  difficult  to  accurately  predict  the  speed  improvement  that  can  be  achieved,  but  a 
factor  or  2  to  4  is  not  unreasonable.  We  base  this  estimate  on  our  experience  with  how  much  time  is 
spent  executing  various  portions  of  each  code  and  realizing  that  some  portions,  such  as  the  basic 
atmospheric  advection,  cannot  be  sped  up  significantly.  The  biggest  improvement  may  be  obtained  by 
restructuring  a  code  to  take  advantage  of  available  hardware  configurations,  such  as  multiple  CPUs. 

5.3.2  Hardware  Improvements. 

Hardware  improvements  continue  to  drive  the  computer  industry;  Moore’s  Law — that  computing  power 
doubles  every  18  months  with  no  increase  in  price — has  been  true  for  the  last  decade  and  is  still  true 
today.  Achieving  improved  execution  speed  through  hardware  selection  is  a  viable  and  cost-effective 
supplement  to  code  improvements.  Furthermore,  to  achieve  the  required  forecast  performance,  we  must 
plan  to  incorporate  more  complex  physical  models,  increasing  the  number  of  floating  point  operations 
and,  hence,  the  demand  on  execution  times.  This  increase  must  be  accomplished  within  the  operational 
constraint  of  one  hour  of  execution  time  for  a  72  hour  forecast. 

Two  existing  workstation  architectures  could  meet  the  TFM  run  time  requirement,  distributed  memory, 
parallel  processor  machines,  such  as  the  IBM  SP-2  and  CONVEX  Exemplar,  or  a  workstation  cluster 
linked  with  a  high  speed  optical  network  interconnect.  These  alternatives  both  use  multiple  CPUs  linked 
by  high-speed  communications,  and  they  differ  primarily  in  packaging;  the  SP-2,  for  example,  consists 
of  several  IBM  RS/6000  CPU  boards  housed  in  a  single  cabinet.  Using  multiple,  independent 
workstations,  either  in  a  single  cabinet  or  free  standing,  also  improves  reliability  and  provides  graceful 
degradation  in  a  field  environment — should  a  single  workstation  element  fail,  the  remaining 
workstations  can  still  produce  a  forecast  with  only  minimal  degradation  in  performance.  Field 
maintenance  can  be  accomplished  at  the  same  level  by  replacing  individual  workstation  elements.  And, 
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because  workstations  have  become  almost  commodity  items,  they  offer  attractive,  competitive  pricing. 
Both  of  these  architectures  offer  a  good  price/perforroance  ratio,  built-in  redundancy,  and  easier 
maintenance  than  other  potential  platforms,  such  as  specialized  vector  computing  machines. 

To  operate  efficiently  on  a  distributed,  multi-processor  system,  a  numerical  weather  prediction  model 
must  be  restructured,  so  it  would  be  advantageous  to  choose  a  model  that  has  already  been  modified  to 
run  on  a  distributed-memory  architecture.  RAMS  and  MM5  have  both  already  been  modified  and  are 
currently  operational  on  workstation  clusters;  NORAPS6  and  RWM  have  not  been  modified.  These 
versions  of  RAMS  and  MM5  have  been  available  and  operating  for  over  two  years.  MM5  was  adapted 
by  the  Scientific  Computing  Division  (SCD)  of  NCAR.  We  have  extensive  experience  running  the 
parallel  version  of  RAMS,  but  not  MM5.  The  following  performance  estimates  are  based  on  our 
experience  with  the  single  processor  versions  of  MM5,  so  the  actual  performance  of  NCAR’s  parallel 
version  may  vary  from  these  estimates.  Using  the  run  times  listed  above,  we  estimate  the  following  run 
times  for  multi-processor  versions. 

In  constructing  Table  5-6,  we  assumed  a  speed  ratio  of  2.5  between  the  IBM  RS/6000-370  and  R.S/6000- 
390.  An  SP-2  consists  of  several  RS/6000-390s  in  a  single  cabinet.  The  third  and  fourth  columns 
assume  1 1  and  14  RS/6000  elements,  respectively.  Our  experience  running  RAMS  on  a  cluster  of  8 
IBM  RS/6000s  shows  that  an  efficiency  factor  of  .75  can  typically  be  achieved.  We  used  this  efficiency 
in  estimating  the  ran  times  on  multiple  workstation  configurations.  Actual  efficiencies  can  depend  on 
the  individual  codes,  the  number  of  computer  elements,  and  the  type  of  communication  used  between 
elements. 

Table  5-6  illustrates  that  using  an  SP-2,  or  similar  computer,  the  distributed  architecture  versions  of 
MM5  or  RAMS  could  meet  the  run  time  requirement.  Using  a  single  element  workstation  will  require 
significant  improvements  in  a  model’s  performance. 

Table  5-6.  Approximate  execution  times  for  a  72  hour  forecast  without  code  improvements. 


RS/6000-370 

RS/6000-390 

1 1  Element 

SP-2 

14  Element 

SP-2 

MM5 

36  hr 

14.4  hr 

<2  hr 

<1.5  hr 

NORAPS6 

8.5  hr 

3.4  hr 

n/a 

n/a 

RAMS 

17.5  hr 

7  hr 

<1  hr 

<45  min. 

RWM 

5.5  hr 

2.2  hr 

n/a 

n/a 
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SECTION  6 

MODEL  ADAPTATION 


6.1  INTRODUCTION. 

The  previously  described  model  comparison  determined  which  mesoscale  forecast  model  best-satisfied 
DSWA  and  AF  accuracy  and  speed  requirements  under  proscribed  resolution  constraints.  For  its 
transport  and  dispersion  program,  DSWA  decided  to  use  RAMS  to  provide  high  resolution 
meteorological  forecasts.  At  the  time  this  report  is  being  written,  the  model  is  being  run  routinely  on  an 
IBM  workstation  at  resolutions  of  10  km  or  less.  A  number  of  changes  have  been  made  in  the  model  as 
a  result  of  adapting  it  to  this  application.  The  majority  of  these  changes  relate  to  either  accepting  data 
from  DSWA  or  military  sources  or  providing  data  in  formats  compatible  with  the  dispersion  models 
used  by  DSWA.  These  formats  are  adequately  described  in  the  User’s  Manual  (Bauer  et  al.,  1997)  being 
delivered  under  this  contract  effort.  Specific  modifications  have  been  made  to  RAMS  to  reduce 
execution  time.  Other  modifications  have  been  explored  and  will  be  used  in  future  versions.  These 
existing  and  potential  changes  will  be  discussed  in  this  section. 

6.1.1  Execution  Time. 

In  evaluating  the  four  candidate  forecast  models  we  emphasized  each  model’s  forecast  performance  and 
accuracy  because  good  performance  is  generally  more  difficult  and  costly  to  achieve  than  is  reduced  run 
time.  However,  run  time  is  important  in  an  operational  environment,  and  the  AF  desired  to  produce  a 
72  hour  forecast  in  1  hour.  DSWA  has  a  similar  objective.  All  tested  models  ran  substantially  slower 
than  this  on  an  IBM  RS/6000  Model  370  workstation.  We  can  meet  the  AF’s  run  time  requirement  by 
using  faster  computers,  by  speeding  up  model  execution,  or  by  a  combination  of  both. 

6.1.2  Hardware  Improvements. 

Hardware  improvements  continue  to  drive  the  computer  industry;  Moore’s  Law — that  computing  power 
doubles  every  18  months  with  no  increase  in  price — has  been  true  for  the  last  decade  and  is  still  true 
today.  Achieving  improved  execution  speed  through  hardware  selection  is  a  viable  and  cost-effective 
supplement  to  code  improvements.  Furthermore,  we  must  anticipate  the  incorporation  of  more  complex 
physical  models,  increasing  the  number  of  floating  point  operations  and,  hence,  the  demand  on  execution 
times.  Several  existing  workstation  architectures  could  meet  the  TFM  run  time  requirement:  distributed 
memory,  parallel  processor  machines,  such  as  the  IBM  SP-2  ,  HP  Exemplar,  SGI  Origin  or  a  workstation 
cluster  linked  with  a  high  speed  optical  network  interconnect.  These  alternatives  all  use  multiple  CPUs 
linked  by  high  speed  communications  with  some  computer  architecture  differences;  the  SP-2,  for 
example,  consists  of  several  IBM  RS/6000  CPU  boards  housed  in  a  single  cabinet,  while  the  SGI  Origin 
200  has  two  enclosures  each  containing  two  processors. 

6.2  RESTRUCTURING  FOR  MULTIPROCESSOR  SYSTEMS. 

To  operate  efficiently  on  a  distributed,  multi -processor  system,  a  numerical  weather  prediction  model 
must  be  restructured.  The  version  of  RAMS  distributed  to  the  worldwide  user  community  at  the  start  of 
this  effort  was  the  single  workstation  version  3b,  the  version  that  was  used  in  the  previously  described 
tests.  Parallel  versions  of  RAMS  capable  of  operating  on  a  parallel  processor  computer  system  were 
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under  development  at  that  time,  and  prototypes  were  under  evaluation  at  Kennedy  Space  Center  (KSC) 
and  elsewhere. 

One  of  the  commonly  available  commercial  parallel  computer  systems  is  the  IBM  SP-2.  The  SP-2  is  a 
Multiple  Instruction,  Multiple  Data  (M3MD)  computer  system.  This  type  of  computer  uses  independent 
workstations  or  nodes  linked  by  a  communications  interface  that  transfers  data  from  one  processor  to 
another.  Therefore,  it  is  basically  a  workstation  cluster  assembled  in  a  single  cabinet  with  enhanced 
communications  and  operating  system.  There  are  several  communication  protocols  that  can  be  used  to 
link  the  various  workstation  nodes  in  this  cluster  environment.  The  communication  protocol  between 
nodes  used  by  the  KSC  prototype  was  the  Parallel  Virtual  Machine  (PVM).  At  the  direction  of  the 
government,  RAMS  was  modified  to  use  the  somewhat  more  recent  Message  Passing  Interface  (MPI) 
communications  protocol.  Essentially  similar  to  PVM,  MPI  is  being  implemented  on  a  number  of 
platforms  including  the  SP-2. 

These  changes  have  been  implemented  in  RAMS  Version  4a.  The  main  change  that  characterizes 
Version  4a  from  the  current  release  version  3b  is  that  the  RAMS  code  structure  has  be  changed  to  allow 
for  the  parallel  computation  on  a  MIMD  computer  architecture,  such  as  an  IBM  SP-2  or  a  workstation 
cluster.  Version  4a  retains  the  ability  to  operate  on  a  single  processor  with  the  same  version  of  the  code. 
This  is  a  user  selectable  option. 

6.2.1  RAMS  Parallel  Version. 

6.2.1. 1  Parallel  Design  Considerations.  Before  beginning  the  modification  of  the  RAMS  code  for 
parallel  computation  in  1991,  we  considered  many  issues.  This  section  will  describe  some  of  these 
considerations.  The  following  were  our  initial  goals;  we  will  mention  if  we  met  these  goals  for  each 
point. 

•  We  concentrated  on  the  “large”  three-dimensional  computational  configurations  of  RAMS,  not 
the  smaller  two-dimensional  runs  that  can  be  requested.  In  general,  the  larger  the  three- 
dimensional  domain  (measured  in  numbers  of  grid  points),  the  more  efficient  the  parallelization 
should  become.  This  goal  has  been  met;  only  three-dimensional  runs  are  allowed  to  run  in 
parallel. 

•  No  compromises  should  be  made  in  the  physics  or  numerics.  It  would  be  possible  to  reduce 
numerical  accuracies  in  some  schemes  such  as  advection  to  reduce  the  amount  of  communication 
among  the  processors.  This  goal  has  been  met,  although  the  sixth-order  advective  scheme  has 
not  been  implemented  yet  in  the  newest  parallel  version. 

•  We  targeted  the  distributed  memory  MIMD  architecture.  With  Single  Instruction,  Multiple  Data 
(SEMD)  computers  or  shared-memory  systems,  a  micro-tasking  structure  in  which  the 
parallelization  is  done  on  the  individual  loop  level  could  be  explored,  which  would  require  a 
distinctly  different  code  structure  than  a  MIMD  machine.  Distributed-memory  architecture 
usually  requires  a  much  coarser-grained  parallelism,  since  communication  between  processors 
needs  to  be  kept  to  a  minimum.  This  goal  has  been  met.  It  has  turned  out  that  the  MIMD 
parallel  structure  has  been  extremely  efficient  on  shared  memory  platforms  tested. 

•  Scalability  of  the  parallel  algorithms  to  massively-parallel  platforms  was  considered.  Although 
our  initial  goal  was  to  target  a  workstation  cluster  (<  8  nodes),  we  have  ported  RAMS  to 
massively-parallel  platforms  (>  64  nodes).  This  goal  has  been  partially  met;  we  are  continuing  to 
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work  on  the  scalability. 

•  For  ease  of  software  maintenance,  a  single  code  version  should  be  developed  for  both 
uniprocessor  and  parallel  platforms.  This  goal  has  been  met 

•  There  should  not  be  any  performance  degradation  for  the  code  on  uniprocessor  platforms  This 
goal  has  been  met  and  we  continue  to  work  on  overall  efficiency  improvements  such  as  better 
cache  usage. 

Therefore,  with  these  requirements  and  considerations,  the  code  modification  has  progressed.  The 
following  sections  will  summarize  the  features  of  the  parallel  RAMS  and  the  basic  techniques  used  in 
accomplishing  the  goals. 

6-2. 1 .2  RAMS  Parallel  Version  Differences.  The  parallel  version  of  RAMS  was  designed  to  be  very 
compatible  with  the  latest  single-processor  version.  The  differences  that  do  exist  between  the  latest 
serial  RAMS  version  (3b)  and  the  parallel  version  are  because  of  unimplemented  features  in  the  parallel 
version  or  at  the  code  level. 

•  Unimplemented  features:  There  are  a  number  of  RAMS  features  that  have  not  been  used  very 
extensively  in  recent  years,  which  were  not  chosen  for  implementation  in  the  parallel  version. 
These  include: 

—  Hydrostatic  equation  set  options 

—  Forward  and  leapfrog  time  differencing  (only  the  “hybrid”  scheme  was  implemented) 

-  “Old”  microphysics  scheme 

•  Code  changes:  At  the  code  level,  there  are  a  large  number  of  changes  mostly  due  to  the 
modifications  for  the  parallel  execution.  The  major  changes  fall  into  four  categories: 

•  Allowing  the  code  to  execute  on  a  sub-domain  -  The  serial  code  was  generally  of  a  structure, 
which  executed  a  process  or  set  of  instructions  over  the  full  set  of  grid  points  for  any  grid.  This 
had  to  be  modified  to  allow  it  to  execute  these  instructions  over  a  subset  of  these  points.  Along 
with  the  considerations  for  parallel  efficiency  (such  as  communication/computation  overlap), 
care  needed  to  be  taken  to  make  sure  that  operations  were  scheduled  correctly  so  that  the 
boundary  regions  of  the  sub-domains  had  access  to  the  correct  information  that  is  passed  from 
adjoining  nodes. 

•  Message  passing  -  Code  was  developed  to  handle  the  passing  of  data  (messages)  between  the 
sub-domains.  Since,  at  different  places  in  the  execution,  different  amounts  of  data  need  to  be 
passed,  different  sets  of  code  were  implemented  to  handle  these  tasks.  Code  to  handle  the 
bookkeeping  chores  also  was  developed.  This  includes  the  tasks  of  domain  decomposition  and 
dynamic  load  balancing. 

•  Additional  C  language  code:  Generally,  the  message  passing  libraries  have  been  developed  and 
written  in  the  C  language.  Therefore,  it  is  advantageous  at  times  to  interface  to  these  libraries 
with  C  routines.  A  few  additional  RAMS  code  files  have  been  rewritten  in  C,  most  notably  the 
main  program.  This  also  allows  for  the  capability  to  include  standard  portable  command  line 
arguments  when  specifying  the  executable  name.  Also,  several  of  the  FORTRAN  modules  are 
now  passed  through  the  C  preprocessor  to  allow  for  conditional  compilation. 

•  Code  improvements  -  Several  improvements  to  the  code  were  made  as  development  of  version 
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4a  progressed,  either  from  efficiency  considerations  or  for  ease  of  use.  Examples  of  these 
changes  include  a  completely  rewritten  advection  scheme  (which  combines  the  leapfrog  and 
forward  advective  schemes  in  the  same  set  of  code)  and  a  new  method  of  setting  up  memory  and 
output  variable  specifications.  Because  of  this  latter  change,  a  new  input  tiara  file  is  now 
required  to  set  up  the  model  variable  structure. 

6. 2. 1.3  Parallel  Components  and  Structure.  The  original  version  of  the  parallel  RAMS,  which  began 
its  development  at  CSU  in  1991,  used  the  Parallel  Virtual  Machine  (PVM)  software,  developed  at  Oak 
Ridge  National  Laboratory,  for  communication  among  the  processors  of  the  parallel  computer  platform 
(Bequelin  et .  ah,  1991).  However,  since  then,  a  consortium  of  industry,  government,  and  university 
computer  scientists  has  developed  a  new  de  facto  standard  called  MPI  (Message  Passing  Interface). 
RAMS  has  been  modified  to  use  MPI  in  addition  to  retaining  the  capability  to  execute  under  PVM. 
RAMS  is  structured  in  a  standard  master-node  configuration,  where  the  master  process  is  a  main 
controlling  process  handling  model  initialization  and  output  while  the  nodes  are  the  main  workers, 
performing  virtually  all  of  the  floating  point  computations  needed  for  the  model  simulations. 


Basic  Code  Structure 

The  basic  structure  of  RAMS  in  this  master-node  configuration  is  the  standard  method  of  domain 
decomposition  where  each  processor  is  given  a  portion  of  the  modeling  domain  on  which  to  perform  the 
computations.  Each  node  is  given  a  rectangular  set  of  grid  points  and  a  surrounding  boundary  region, 
which  will  be  referred  to  as  the  subdomain.  Again,  all  computation  is  done  on  the  node  processes. 
During  each  model  timestep,  the  nodes  must  exchange  information  at  the  subdomain  boundaries. 

Domain  decomposition 

Domain  decomposition  is  the  process  of  spatially  subdividing  any  computational  grid  domain  in  the 
model  into  two  or  more  subdomains,  each  one  to  be  handed  to  a  separate  processor  for  carrying  out  the 
necessary  computations  for  time  integration.  There  are  many  possible  ways  of  decomposing  a  3-D  grid 
domain,  but  the  following  considerations  point  strongly  to  one  best  method. 

•  Some  computational  algorithms  in  the  model  solve  tri-diagonal  linear  systems  in  the  vertical, 
requiring  simultaneous  knowledge  of  all  grid  cell  values  in  the  vertical  column.  Thus,  it  is  to 
great  advantage  to  keep  an  entire  column  on  the  same  processor  and  to  decompose  only  in  one  or 
both  horizontal  directions. 

•  Although  1-D  horizontal  decomposition,  i.e.,  grouping  not  only  an  entire  column  but  also  all 
columns  in  a  given  constant-x  or  constant-y  slab  into  the  same  subdomain,  is  conceptually  and 
algorithmically  simpler  than  2-D  decomposition,  it  increases  both  the  total  memory  requirement 
and,  more  importantly,  the  amount  of  information  that  must  be  communicated  between 
processors.  Thus,  we  adopted  the  more  general  approach  of  decomposing  in  both  horizontal 
directions. 

•  We  desire  the  flexibility  to  utilize  any  number  of  processors  that  may  be  available  including 
prime  numbers. 

•  Parallel  efficiency  dictates  that  all  processors  should  perform  their  computational  tasks  in  nearly 
the  same  amount  of  time  so  that  no  processor  needs  to  waste  time  waiting  for  the  results  from 
another.  This  requirement,  coupled  with  the  facts  that  (1)  processors  may  run  at  different  speeds 
(e.g.,  in  a  parallel  cluster  of  dissimilar  workstations)  and  (2)  certain  regions  of  a  model  grid 
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require  more  computational  steps  than  others  (e.g.,  where  clouds  occur),  dictates  that  model  grid 
subdomains  must  be  allowed  to  have  unequal  sizes  in  order  to  properly  balance  the 
computational  load  on  each  processor. 

The  domain  decomposition  algorithm  in  RAMS  satisfies  all  the  above  requirements.  The  algorithm 
requires  as  input  the  number  of  grid  points  (vertical  columns)  in  each  horizontal  direction,  the  number  of 
processors  to  be  used,  the  relative  speed  of  each  processor,  and  a  set  of  “column  work  factors”  or  relative 
computational  workload  (CPU  time)  of  each  vertical  column.  Both  the  relative  processor  speeds  and 
column  work  factors  may  be  evaluated  from  the  performance  over  the  preceding  timesteps  or,  at  the 
beginning  of  a  model  ran,  estimated. 

A  diagram  of  grid  subdomains  resembles  bricks  laid  in  tiers,  with  straight,  unbroken  lines  separating 
each  tier.  The  lines  separating  the  bricks  in  each  tier  generally  terminate  at  the  top  and  bottom  of  the  tier 
but  in  some  cases  coincide  with  lines  in  adjacent  tiers.  The  number  of  tiers  is  an  integer  close  to  the 
square  root  of  the  number  of  processors,  as  is  the  number  of  bricks  in  each  tier,  but  adjustments  are  made 
as  necessary  to  accommodate  the  exact  number  of  processors  available.  Different  tiers  in  general  have 
different  thicknesses,  and  bricks  in  a  tier  in  general  have  different  lengths.  These  dimensions,  i.e.,  the 
horizontal  dimensions  of  each  subdomain,  are  collectively  determined  from  the  relative  processor  speeds 
and  the  column  work  factors  in  order  to  achieve  precise  load  balancing. 

An  entire  model  grid  consists  of  interior  vertical  grid  columns  where  field  variables  are  prognosed  plus  a 
single  row  of  grid  columns  in  which  diagnostic  lateral  boundary  conditions  are  applied  forming  a  lateral 
perimeter.  In  the  domain  decomposition,  each  interior  prognostic  column  is  placed  in  only  one 
subdomain  and  prognosed  only  in  that  subdomain  which  completely  avoids  repetitive  prognostic 
computations  between  different  processors.  A  single  row  of  grid  columns  (the  “halo”  or  overlap  region) 
is  added  to  form  a  perimeter  around  each  subdomain  prognostic  region,  which  duplicates  a  modest 
number  of  columns  comprising  the  entire  grid.  These  columns  are  used  to  store  field  values 
communicated  from  adjacent  subdomain  interiors  where  they  are  prognosed. 

In  applications  where  nested  grids  are  employed  in  RAMS,  domain  decomposition  is  completely 
independent  for  each  grid.  Thus,  decomposition  is  based  only  on  the  grid  size  and  work  factors  plus  the 
number  and  speed  of  the  processors  and  does  not  depend  on  where  the  grid  is  placed  within  its  parent 
grid,  where  finer  grids  are  placed  within  it,  or  how  the  finer  and  parent  grids  are  decomposed. 

Types  of  communication 

There  are  seven  distinct  types  of  message-passing  events  in  the  RAMS  structure.  These  are: 

•  Initialization:  During  initialization,  the  master  process  will  compute  the  grid  decomposition  and 
send  all  necessary  information  and  data  to  the  compute  nodes.  The  full  subdomain  of 
information  is  sent  for  almost  all  variables.  This  will  also  occur  during  the  dynamic  load 
balancing. 

•  Long  timestep  overlap  region:  At  the  beginning  of  a  timestep,  the  nodes  will  exchange  the 
overlap  regions  of  the  prognostic  variables.  We  have  reduced  the  overlap  region  to  one  row. 

•  Long  timestep  overlap  region  (turbulence):  In  order  to  reduce  the  overlap  region  to  one  row,  it 
was  necessary  to  add  an  extra  communication  step  where  the  turbulence  exchange  coefficients  in 
the  overlap  region  are  exchanged  between  nodes.  This  technique  has  added  an  additional  5-10% 
parallel  efficiency,  depending  on  the  platform,  to  the  RAMS  runs. 
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•  Small  timestep  overlap  region:  During  the  computation  of  the  small  acoustic  timestep,  the 
compute  nodes  will  exchange  the  overlap  regions  of  the  u  and  v  velocity  components  and  the 
pressure.  This  is  only  needed  for  one  overlap  row. 

•  Nested  grid  boundaries:  Nodes  containing  a  coarse  grid  will  send  the  necessary  data  to  a  fine  grid 
node  for  interpolation  of  the  nested  grid  boundaries. 

•  Nested  grid  feedback:  The  compute  nodes  will  average  their  fine  grid  information  to  the  coarse 
grid  structure  and  transfer  that  data  to  the  appropriate  coarse  grid  nodes. 

•  File  output:  When  it  is  time  for  file  output,  the  compute  nodes  will  send  the  full  subdomain  of 
the  prognostic  variables  to  the  master  process. 

All  node  to  node  communication  types  package  variables  into  a  single  message  before  sending  to  the 
receiving  nodes. 

Concurrent  computation/communication 

It  is  desirable  that  communication  between  each  processor  of  the  parallel  system  be  kept  to  a  minimum  if 
the  overhead  involved  in  the  communication  between  processors  in  a  system  is  significant  compared  to 
the  computations  performed.  One  way  of  minimizing  the  communication  cost  is  to  “schedule” 
communications  so  that  they  occur  concurrently  with  computation.  The  modular  structure  of  the  RAMS 
code  is  very  conducive  to  this  type  of  technique.  The  timestep  computational  structure  has  been 
rearranged  so  that  many  of  the  model  computational  schemes  that  do  not  require  information  from  the 
subdomain  boundary  regions  (those  schemes  that  operate  either  at  a  single  grid  point  or  in  a  vertical 
column)  are  executed  first  during  a  timestep.  In  addition,  some  of  the  model  routines  compute  the 
interior  portion  of  subdomains  first.  While  these  are  occurring,  the  node  processes  exchange  the 
subdomain  boundary  information.  When  the  node  is  finished  with  the  first  set  of  computations,  the 
subdomain  boundary  information  will  have  been  received  so  that  the  node  can  continue  with  the 
remainder  of  the  computations.  For  smaller  number  of  nodes,  the  concurrent  scheduling  of 
communication  and  computation  can  even  allow  for  reasonable  efficiency  in  the  use  of  a  cost-effective, 
standard  Ethernet  for  communication  in  a  clustered  workstation  system  (over  a  small  number  of  nodes), 
rather  than  the  more  expensive  optical  fiber  components. 

Bookkeeping  arrays 

Once  the  domain  decomposition  is  performed,  all  required  communications  between  subdomain 
processors  are  determined.  This  involves  tabulating  the  exact  set  of  grid  point  locations  for  each 
variable  to  be  sent  and  received  between  all  relevant  pairs  of  subdomains.  In  order  to  minimize 
communication,  only  required  variables  from  required  locations  are  sent  and  received.  In  some  cases, 
only  a  single  grid  column  needs  to  be  communicated  between  processors  while  in  others  a  rectangular 
block  of  columns  must  be  sent.  The  starting  and  ending  grid  coordinates  in  both  horizontal  directions 
for  each  block  are  stored  in  a  5-D  integer  array.  The  arguments  of  this  array  are  (1)  the  source  or  sending 
subdomain  or  processor,  (2)  the  destination  or  receiving  subdomain  or  processor,  (3)  the  model  grid 
number  (since  decomposition  is  different  in  general  for  each  grid),  (4)  the  designation  of  one  of  the  four 
block  coordinates,  and  (5)  the  type  of  communication. 

File  output 

The  file  output  of  RAMS  is  accomplished  through  the  master  process.  When  the  time  comes  to  output 
the  files,  the  compute  processes  send  all  necessary  information  back  to  the  master,  which  can  then  write 
the  files  to  local  disk.  Other  techniques  have  been  used  where  each  node  will  output  its  portion  of  the 
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data,  then  a  separate  process  will  recombine  the  individual  parts.  The  concept  of  the  master  I/O  process 
has  several  advantages  including: 

•  The  transfer  of  data  is  done  over  the  usually  higher  speed  message  passing  hardware.  Network 
File  System  software  is  not  used. 

•  No  local  disk  space  is  needed  (for  MIMD  platforms). 

•  After  the  nodes  have  transferred  their  data,  they  can  continue  with  their  processing  without 
waiting  for  the  master  process  to  finish  writing  the  files. 

•  The  output  files  are  immediately  available  from  the  master  node  for  other  activities. 


Nested  grid  considerations 

As  mentioned,  RAMS  contains  a  two-way  interactive  grid  nesting  scheme  where,  during  a  timestep,  the 
coarser  grid  gives  boundary  information  to  a  fine  grid  and  a  fine  grid  averages  its  domain  and  overlays 
the  appropriate  area  of  the  coarse  grid.  Nested  grids  can  be  either  telescoping  or  simultaneously  nested 
within  a  coarser  grid.  There  is  no  limit  imposed  in  RAMS  as  to  the  number  of  nests  or  the  spatial 
nesting  ratio  between  nests.  This  situation  does  pose  a  challenge  for  the  development  of  an  efficient 
parallel  version  of  RAMS  which  uses  domain  decomposition,  dynamic  load  balancing,  and  strives  to 
minimize  communication  among  the  nodes. 

The  nesting  scheme  has  been  modified  to  increase  its  scalability.  The  new  scheme  decomposes  each  of 
the  nested  grids  independently.  Then  the  appropriate  communications  are  done  for  the  two-way  nesting 
algorithms.  For  the  feedback  process,  the  fine  grid  data  is  averaged  on  the  fine  grid  nodes,  then  . 
transferred  to  the  coarse  nodes.  For  the  nested  boundary  interpolations,  the  coarse  grid  nodes  send  the 
data  to  the  fine  grid  nodes  where  the  interpolation  is  done.  Note  that  in  each  of  these  schemes,  only 
coarse  grid  data  is  transferred,  thus  reducing  the  amount  of  communication  as  much  as  possible. 

Dynamic  load  balancing 

With  the  techniques  of  domain  decomposition,  inefficiencies  will  arise  when  one  processor  takes  much 
longer  with  its  subdomain  than  the  other  processors.  This  can  happen  frequently  in  RAMS  because  of 
spatial  differences  in  model  physics  and  hence  the  complexity  of  the  physical  parameterizations  the 
model  has  to  compute.  If  this  happens,  the  remainder  of  the  processors  will  sit  idle.  However,  we  have 
implemented  a  technique  for  dynamic  load  balancing,  an  objective  way  to  allow  for  the  adjustment  of 
the  computational  load  among  the  processors  as  a  model  simulation  progresses. 

The  dynamic  balancing  in  RAMS  is  accomplished  in  the  following  manner.  At  the  end  of  each  timestep, 
the  compute  nodes  will  send  the  CPU  time  and  the  wall  clock  time  from  its  subdomains  back  to  the 
master  process.  The  master  process  then  can  determine  if  imbalances  are  occurring.  When  the  compute 
nodes  have  sent  all  the  subdomain  fields  back  to  the  master  process,  which  occurs  when  it  is  time  to 
write  an  output  file  to  disk,  the  master  process  is  able  to  basically  repeat  the  initialization  procedure. 

This  consists  of  computing  the  work  factor  for  each  grid  column,  decomposing  the  model  grids 
according  to  the  work  factors,  and  sending  out  new  subdomain  information  and  data  to  the  compute 
processes. 


225 


6.2. 1.4  Parallel  Efficiency  Results.  Numerous  tests  have  been  run  with  the  parallelized  version  RAMS 
on  several  parallel  platforms.  As  of  this  writing,  we  have  ported  the  code  to  workstation  clusters,  the 
IBM  SP2,  and  the  HP/Convex  Exemplar.  Other  groups  have  ported  the  parallel  RAMS  to  the 
CRAYJ90,  T3D,  and  T3E  and  the  SGI  Power  Challenge.  Obviously,  the  parallel  performance  is 
dependent  on  the  hardware  characteristics  including  the  machine  architecture,  speed  of  CPU,  and  type 
and  speed  of  the  CPU  interconnect.  But  parallel  efficiency  is  also  dependent  on  the  configuration  of  the 
model  simulation,  with  such  things  as  numbers  of  grid  points,  number  of  grids,  and  complexity  of 
physics  needing  to  be  considered. . 


For  the  ports  we  have  made,  we  will  state  generally  some  of  the  parallel  efficiencies  we  have  attained. 

•  Workstation  clusters:  We  have  run  on  several  clusters  (2-8  nodes)  of  IBM  RS/6000  workstations 
using  standard  Ethernet  as  the  networking  hardware.  Efficiencies  attained  have  ranged  from  60% 
to  85%  depending  on  model  configuration  and  number  of  nodes.  Efficiency  is  defined  as: 

_ time  required  for  a  single  processor _ 

(time  required  for  the  parallel  processor)  (number  of  processors) 

•  IBM  SP2:  On  the  SP2  using  the  high-speed  switch,  efficiencies  have  ranged  from  about  90%  on 
larger  simulations  using  8  nodes  to  about  68%  using  64  nodes. 

•  HP/Convex  Exemplar:  The  Exemplar  (or  SPP  1600)  is  an  8  processor,  shared  memory  machine 
using  a  crossbar  switch.  For  large  multi-grid  runs,  we  routinely  achieve  90-95%  efficiency  on  the 
8  processors.  For  moderate  size  multi-grid  runs  (number  of  horizontal  grid  points:  50  by  50),  we 
are  able  to  get  better  than  100%  efficiency  because  of  better  cache  utilization. 

•  SGI  Origin:  We  have  access  to  a  Silicon  Graphics  Origin  200,  a  4  processor,  shared  memory 
machine.  For  medium-size,  multi-grid  runs,  we  have  achieved  90-95%  efficiency. 

6.2.1 .5  Future  Developments.  RAMS  continues  to  be  developed  and  capabilities  expanded. 

Following  are  some  of  the  issues  we  will  be  considering  in  the  near  future  as  related  to  the 
computational  performance. 

•  Increased  serial  code  efficiency:  We  continue  to  look  for  ways  to  increase  the  single  processor 
performance  of  the  code.  These  include  better  algorithmic  ways  to  program  various  numerical 
techniques  and  to  better  utilize  the  CPU  cache. 

•  Investigate  additional  concurrent  communication/computation  possibilities:  Additional 
possibilities  exist  for  overlapping  the  computations  and  communications  such  as  computing  more 
terms  on  the  sub-domain  interior  first. 

•  More  complicated  "work"  factor:  The  computation  of  the  work  factor,  which  is  used  in  the 
domain  decomposition,  is  not  a  straightforward  technique,  as  it  relies  on  the  presence  of  a  wide 
range  of  different  variables  and  the  knowledge  of  how  these  variables  affect  the  computational 
cost.  We  are  continuing  to  experiment  with  better  algorithms  to  define  this  quantity. 

•  Continue  to  explore  shared-memory  architecture  considerations:  Computer  manufacturers  are 
beginning  to  introduce  MIMD-type  platforms  where  each  node  is  a  symmetric  multi-processor. 
We  will  be  looking  at  this  architecture  to  determine  the  most  efficient  ways  to  utilize  these 
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platforms.  Also,  several  shared  memory  platforms  with  a  moderate  number  of  processors  (8-32 
CPUs)  have  come  on  to  the  market  recently.  Although  our  initial  impression  is  favorable  as  to 
the  efficiency  of  a  MIMD  code  structure  on  these  platforms,  we  will  continue  to  investigate  the 
code  performance. 

6.3  CODE  IMPROVEMENTS. 

We  ran  each  of  the  candidate  forecast  models  using  the  best  physics  models  each  one  offered.  It  is 
unrealistic  to  relax  this  choice  because;  even  using  their  best  physics,  none  of  the  models  fully  met  the 
AF’s  forecast  performance  criteria.  In  fact,  the  converse  would  appear  to  be  true:  additional  and  better 
physics  models  must  be  added  to  improve  forecast  ability  and  to  predict  phenomena,  such  as  clouds,  that 
are  not  provided  by  current  models. 

In  general  it  is  more  cost-effective  to  speed  up  existing  physics  models  than  to  develop  new  models.  We 
can  improve  the  performance  of  existing  detailed  physical  models  by  parameterizing  their  output  or  by 
approximating  parts  of  them  with  analytic  functions.  Of  the  four  TFM  candidates,  RAMS  and  MM5 
could  benefit  most  from  this  type  of  improvement  because  they  already  contain  the  most  sophisticated 
physics  models.  It  is  difficult  to  accurately  predict  the  speed  improvement  that  can  be  achieved,  but  a 
factor  or  2  to  4  is  not  unreasonable.  We  base  this  estimate  on  our  experience  with  how  much  time  is 
spent  executing  various  portions  of  each  code  and  realizing  that  some  portions,  such  as  the  basic 
atmospheric  advection,  cannot  be  sped  up  significantly.  Speeding  up  individual  physics  models  is 
generally  a  difficult  and  time-consuming  process.  However,  late  in  the  contract  a  technique  called  the 
Fully  Equivalent  Operational  Model  (FEOM)  was  identified  as  a  potential  means  of  improving 
performance  significantly  without  laborious  recoding. 

6.3. 1  Investigation  To  Reduce  Physics  Model  Execution  Time. 

We  investigated  methods  for  reducing  the  time  required  to  run  the  RAMS  model  would  allow  weather 
and  dispersion  predictions  to  be  made  quicker  and  updated  more  frequently.  Traditional  methods  for 
reducing  computational  runtimes  trade  model  prediction  fidelity  for  reduced  ran  times  through  model 
simplifications.  Here  we  document  an  innovative  approach  to  reduce  RAMS  runtime  while  preserving 
prediction  fidelity.  This  involved  the  application  of  a  new  computational  modeling  tool,  the  Fully 
Equivalent  Operational  Model  (FEOM)  algorithm,  to  a  computationally  intensive  radiation  transport 
module  in  the  RAMS  code.  The  FEOM  algorithm  produced  a  new  module  that  predicts  the  original 
module’s  output  with  high  precision  and  103  times  faster.  The  new  algorithm  was  developed  by  Dr. 
Jeffrey  A.  Shorter  of  Mission  Research  Corporation  and  Professor  Herschel  Rabitz  of  Princeton 
University. 

The  two  modules  that  consume  the  greatest  portion  of  computational  resources  in  RAMS  are  cloud 
microphysics  and  Radiation  Transport  Module  (RTM).  The  microphysics  module  calculates  the 
evolution  of  rain,  ice,  and  snow  in  the  model  while  the  RTM  calculates  the  net  amount  of  energy 
deposited  into  the  atmosphere  from  solar  radiation,  blackbody  emission  from  ground,  and  infrared 
emission  from  the  atmosphere.  The  proof  of  concept  for  the  conversion  of  the  RTM  into  a  FEOM  has 
been  completed  and  is  described  here.  While  time  and  effort  did  not  allow  a  complete  development  of 
the  RTM  and  its  implementation  into  RAMS,  the  results  are  very  encouraging. 

6.3 . 1 . 1  Background  -  Radiative  Transfer  Module.  The  RTM  in  RAMS  calculates  altitude  dependent 
atmospheric  heating  rates  due  to  absorption  and  emission  of  solar  and  infrared  energy . 
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The  heating  rate  is  used  to  predict  changes  to  the  atmospheric  temperature  profiles.  A  physics-based 
two-stream  radiation  transport  model  is  used  for  the  calculations.  This  model  treats  the  atmosphere  as  a 
series  of  horizontal  layers.  Solar  light  impining  from  the  top  of  the  atmosphere  is  propagated  layer  by 
layer  through  the  atmosphere,  then  reflects  off  the  Earth's  surface,  and  propagates  up.  layer  by  layer.  In 
each  layer,  the  light  can  pass  through,  be  absorbed,  or  scattered  by  molecules  and/or  aerosols.  Scattering 
is  either  in  the  forward  or  reverse  direction.  The  infrared  longwave  emission  from  the  Earth  is 
propagated  upward  layer  by  layer  and  can  either  pass  through  the  layer  or  be  absorbed.  Since  long 
wavelength  energy  from  blackbody  radiation  and  infrared  emission  is  not  treated  in  the  model.  The 
infrared  emission  of  three  trace  gases  (H20,  03  and  C02)  are  treated  in  this  model.  This  molecular 
emission  is  propagated  both  upward  and  downward  in  the  atmosphere  and  can  either  pass  either  through 
the  layer  or  be  absorbed.  These  processes  are  shown  schematically  in  Figure  6-1 .  The  horizontal  lines 
represent  the  layering  of  the  atmosphere.  The  set  of  arrow's  on  the  left  depict  the  solar  radiation  entering 
the  atmosphere  from  above,  propagating  downward,  reflecting  off  the  surface,  and  propagating  back 
upward.  The  middle  set  of  arrow's  represents  the  black  body  emission  from  the  Earth  that  is  propagated 
upward.  The  set  of  arrows  on  the  right  depicts  the  infrared  emission  from  the  trace  gas  species  in  one 
layer  going  in  the  upward  and  downward  directions.  This  light  is  treated  similarly  to  the  black  body 
emission.  In  the  first  two  cases,  the  weight  of  the  arrows  represents  the  magnitude  of  the  optical 
intensity.  As  the  lines  narrow,  the  light  is  being  absorbed  by  the  atmosphere  and  transferred  to  heat. 

The  RTM  calculates  the  amount  of  light  absorbed  and  converts  it  to  a  heating  rate  (deg/sec).  The 
atmospheric  trace  gas  concentrations  are  either  prescribed  by  McLatchy  soundings  (03  and  C02)  or  are 
prognostically  calculated  in  RAMS  (H20). 

The  evolution  of  atmospheric  dynamics  is  very  sensitive  to  the  heating  rates  calculated  in  the  RTM.  As 
such,  a  first  principle  approach  is  used.  This  involves  separating  the  radiation  into  many  different 


Figure  6-1.  Schematic  of  the  atmospheric  representation  in  the  radiation  transport  module. 
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wavelength  bands,  calculating  wavelength-dependent  absorption  coefficients  for  each  trace  gas  absorber, 
and  coupling  the  downward  and  upward  radiation  fluxes.  As  a  result,  the  RTM  is  very  computationally 
intensive.  To  compensate  for  the  RTM  runtime,  RAMS  only  calls  the  module  every  10  to  30  time  steps 
and/or  only  at  specific  locations  in  the  grid  space.  The  heating  rates  are  assumed  not  to  change  between 
RTM  calculations.  For  locations  in  the  grid  between  RTM  calculation  points,  the  heating  rates  are 
interpolated.  Even  with  this  restricted  utilization  of  the  RTM,  it  consumes  between  15-30%  of  the  total 
CPU  time  of  a  simulation. 

By  calculating  heating  rates  orders  of  magnitude  faster,  while  maintaining  accurate  results,  one  could: 

•  Run  the  RTM  more  frequently  to  increase  the  temporal  fidelity  of  the  heating  rate  profiles; 

•  Run  the  RTM  for  more  grid  columns  to  increase  the  spatial  fidelity  of  the  heating  rate  profiles; 

•  Reduce  the  overall  runtime  of  RAMS  by  one  third  (estimated). 

This  acceleration  of  the  RTM  has  been  achieved  through  the  use  of  FEOMs  and  is  discussed  below. 

6. 3. 1.2  Fully  Equivalent  Operational  Models  (FEOMs).  FEOM  addresses  the  perceived  complex 
problem  of  mapping  the  input-output  relationship  behavior  of  complex  systems.  A  traditional  approach 
to  a  problem  with  n  input  variables  x„. .  .,x„  would  consist  of  sampling  each  variable  at  s  point  to 
assemble  an  interpolated  lookup  table  of  computational  effort  s".  Realistically  one  may  expect  s  to  be 
approximately  5  to  10  and  n  can  be  10-1 02  or  larger.  Viewed  from  this  perspective,  attempts  at  creating 
a  lookup  table  would  be  prohibitive.  However,  this  analysis  implicitly  assumes  that  all  n  variables  are 
important  and,  most  significantly,  that  there  are  correlations  among  variables  to  all  orders  (i.e.,  single 
pairs,.. . .,  up  to  all  n  variables  acting  in  a  tightly  correlated  fashion). 


The  fundamental  principle  underlying  the  FEOM  representation  is  that  output  correlations  between 
independent  variables  die  off  rapidly.  This  assertion  does  not  eliminate  strong  variable  dependence  or 
even  the  possibility  that  all  the  variables  are  important.  Various  sources  of  information  support  this 
point.  First,  variables  in  most  physical  models  enter  as  simple  functions,  often  as  additive  sums  of  low- 
order  multiplicative  terms.  This  kinematic  simplicity  tends  to  survive  in  the  output,  although  often 
scrambled  in  a  complex  fashion.  Second,  traditional  statistical  analysis  of  model  behavior  has  revealed 
that  a  variance  and  covariance  analysis  of  the  output  in  relation  to  the  input  variables  often  adequately 
describes  the  physics  (i.e.,  only  low-order  correlations  describe  the  dynamics).  Finally,  these  statements 
are  also  familiar  from  many-body  physics  where  rarely  do  interactions  higher  than  two-body  play  a 
significant  role.  This  general  observation  leads  to  a  dramatically  reduced  computational  scaling  when 
one  seeks  to  map  input-output  relationships  of  complex  systems.  Considering  this  analysis,  one  may 
now  argue  power-law  scaling  to  learn  the  input-output  behavior,  such  as  ~n'.  Here,  /  is  the  highest  order 
of  significant  correlation  and,  typically,  /  <  3.  Polynomial  growth  poses  a  far  more  tractable  algorithm 
than  the  often  accepted  view  of  exponential  growth  sn  above. 

The  approach  to  generating  a  FEOM  is  to  evaluate  the  input-output  response  of  the  model.  This  is 
achieved  by  performing  a  hierarchical,  correlated  function  expansion  of  a  special  mathematical  structure 
on  the  model  and  evaluating  each  term  of  the  expansion  independently.  This  expansion  is  then  used  as 
the  FEOM.  One  may  show  that  a  function,  g(x)=g(xt,X2,...,xn)  can  uniquely  be  decomposed  into 
summands  of  different  dimensions: 
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(6.1) 


g(x,,x2,...,xD)  =  /0  +X /,(*,)+  .2.3, — n  (XMX2’***’Xn) 

1=1  iSiZj^n 

where /0  is  a  constant, /i(xO  is  a  function  of  one  variable  xx,fij(x»Xj)  is  a  function  of  two  variables,  x;,  xj, 
and  the  higher  dimensions  follow  until /ii3,...J1(xi,x2,X3,...^n)  which  is  a  function  of  all  the  variables  (i.e., 
it  is  defined  as  being  the  remainder:  g  minus  all  the  prior  terms  in  the  sum).  The  function  yj(x;) 
describes  the  independent  action  of  the  variable  Xj,  while ^(x^xj)  gives  the  pair  correlated  behavior,  etc. 

By  representing  the  function  as  a  summation  of  terms  and  investigating  their  respective  magnitudes,  it  is 
expected  that  many  will  not  contribute  significantly  to  the  summation  and  can  be  removed.  This  is  a  key 
benefit  to  the  FEOM  approach.  Furthermore,  this  representation  makes  interpolating  between  function 
values  trivial  since  they  are  low  dimensional  interpolations.  Importantly,  no  constraining  forms  are 
applied  to  the  expansion  functions;  they  are  numerically  represented  and  form  an  interpolate  look-up 
table. 

The  individual  functions  are  explicitly  calculated  by  running  the  model  a  number  of  times  with  a  series 
of  judiciously  chosen  input  sets.  Special  care  is  taken  to  ensure  that  the  entire  phase  space  of  the  model 
input  is  covered  when  generating  the  FEOM.  The  FEOM  generation  process  is  shown  schematically  in 
Figure  6-2.  A  series  of  input  sets  are  generated  and  run  through  the  model.  The  resulting  output  is  then 
related  back  to  the  input  and  the  input-output  relationship  is  deduced.  This  is  then  encapsulated  into  the 
FEOM. 

6.3.1 .3  FEOM  Construction.  The  proof  of  concept  for  converting  the  RTM  into  a  FEOM  was 
achieved. 


Figure  6-2.  Schematic  of  the  FEOM  generation  process. 
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The  FEOM  developed  for  the  RTM  calculates  heating  rates  between  0  and  20  km  altitude  in  30  discrete 
levels.  The  predictions  are  within  0.1%  of  RAMS  RTM  calculated  output  values  on  average  and  the 
FEOM  predictions  are  made  103  times  faster.  The  inputs  to  the  FEOM  are  H20  concentrations  at  each 
level,  the  atmospheric  temperature  at  each  level,  the  surface  temperature,  and  the  surface  albedo.  A  clear 
sky  condition  was  assumed  for  this  initial  investigation  and  the  solar  input  flux  at  the  top  of  the 
atmosphere  was  assumed  to  be  constant.  Both  of  these  assumptions  could  be  incorporated  with  little 
impact  on  the  overall  FEOM  runtime.  A  schematic  of  the  RTM  FEOM  is  shown  in  Figure  6-3.  The 
FEOM  uses  the  same  input  parameters  as  the  RTM  and  predicts  the  same  output.  This  makes  the 
implementation  of  the  FEOM  transparent  to  RAMS.  A  series  of  discrete  layers  are  represented  in  the 
heating  rate  profile  output  box  to  emphasize  that  the  FEOM  actually  consists  of  a  set  of  sub-FEOMs 
with  each  one  predicting  the  heating  rate  in  a  given  layer.  The  FEOM  approach  can  be  constructed  to 
allow  the  user  flexibility  of  defining  altitude  ranges  and  number  of  layers  even  though  this  particular 
FEOM  is  for  a  specific  number  of  layers  and  altitude  ranges. 


Figure  6-3.  Schematic  of  the  radiative  transfer  module  FEOM. 


Input  parameter  phase  space. 

The  H2O  and  air  temperature  profiles  are  represented  as  deviations  from  a  reference  mid-latitude 
summer  profile.  The  profiles  are  broken  into  30  discrete  layers  spaced  0.666  km  apart  and  span  an 
altitude  range  from  0  to  20  km.  In  each  layer,  the  H2O  value  can  vary  from  10%  to  160%  of  the 
reference  value.  The  temperature  can  vary  ±  10%  from  the  reference  values.  We  emphasize  that  the 
H2O  and  temperature  values  in  each  layer  are  treated  independent  of  from  all  other  layers  and  thus  the 
profiles  can  be  very  structured.  The  H2O  and  temperature  ranges  are  shown  in  Figure  6-4.  The  surface 
temperature  can  vary  from  -8  to  50°C  and  the  surface  albedo  can  vary  from  0  to  1.0. 
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Figure  6-4.  Reference  input  profiles  for  H20  and  temperature  (left).  The  heating  rate  calculated 
using  the  RAMS  RTM  (right). 


Table  6-1  summarizes  the  description  of  the  parameters  as  well  as  the  minimum  and  maximum 
variation. 


Table  6-1.  List  of  input  parameters  for  the  FEOM. 


Parameter 

Description 

Minimum 

Maximum 

1-30 

H20  for  Level  1  to  30 

0.1 

1.6 

31-60 

Air  Temperature  for  Level  1  to  30 

0.9 

1.1 

61 

Surface  Temperature  (°C) 

-8 

50 

62 

Surface  Albedo 

0.0 

1.0 

Radiative  Transfer  Module  Output. 

The  RTM  calculates  the  altitude  dependent  atmospheric  heating  rate  in  degrees  per  second.  The  heating 
rate  can  be  either  positive,  indicating  that  the  atmospheric  layer  is  absorbing  more  energy  than  it  is 
emitting,  or  negative,  indicating  more  optical  emission  than  absorption.  The  values  typically  range 
between  ±  4xl0"5  K/sec  (±  3.5K/day).  The  heating  rate  profile  for  the  mid-latitude  reference  atmosphere 
is  shown  in  Figure  6-4. 
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Functional  Representation. 

A  FEOM  to  replace  the  RTM  will  consist  of  a  set  of  sub-FEOMs,  one  for  each  vertical  layer  in  the 
model.  Each  sub-FEOM  predicts  the  heating  rate  for  a  specified  atmospheric  layer.  Since  the 
atmospheric  representation  for  this  proof  of  principle  was  divided  into  thirty  layers,  thirty  independent 
FEOMs  were  developed.  Though  each  sub-FEOM  only  predicts  the  heating  rate  for  a  specified  layer,  it 
uses  information  from  all  other  layers  to  make  the  prediction.  This  infonnation  is  contained  in  the  RTM 
input  parameters:  30  H2O  values  (one  for  each  layer),  30  air  temperatures,  the  surface  temperature,  and 
the  surface  albedo. 

For  each  of  the  sub-FEOMs,  the  first  three  sets  of  terms  of  the  function  expansion  (Equation  6.1)  were 
calculated.  There  is  one  zeroth  order  term,  which  is  determined  by  running  the  mcidel  with  the  nominal 
values  of  the  model.  There  are  62  first  order  term  functions,  one  for  each  parameter,  and  1891  second 
order  terms,  one  for  each  possible  pair  of  parameters.  Each  first  order  function  was  represented  with  16 
points  spanning  the  parameter  phase  space.  Each  second  order  function  was  represented  by  16  points  for 
each  parameter,  resulting  in  a  16  x  16  surface  (256  points). 

Once  all  1,954  functions  were  calculated  for  each  sub-FEOM,  they  were  evaluated  for  significance.  If  a 
function  is  not  significant,  it  is  removed  from  the  expansion.  The  ability  to  remove  these  functions  is  an 
advantage  of  the  FEOM  function  expansion’s  special  mathematical  structure.  A  function  was 
considered  significant  if  the  absolute  magnitude  of  any  point  of  the  surface  was  greater  than  lxlCT6 
deg/sec.  This  value  is  approximately  2%  of  the  maximum  heating  rate  magnitude.  Using  this  criteria, 
304  first  order  functions  and  138  second  order  functions  for  all  30  sub-FEOMS  were  kept.  That  is  a 
very  small  fraction  of  all  the  first  and  second  order  functions  evaluated.  The  number  of  first  and  second 
order  terms  kept  for  each  layer  is  shown  in  Table  6-2. 

6.3 . 1 .4  Analysis  of  the  FEOM.  The  completed  FEOM  was  evaluated  for  prediction  accuracy  and 
runtime  relative  to  the  original  RAMS  RTM.  The  evaluation  was  performed  by  running  both  models 
over  a  range  of  input  conditions  and  comparing  both  the  output  results  and  the  time  required  to  attain  the 
results.  The  input  sets  were  generated  using  Monte  Carlo  methods  such  that  each  input  parameter  was 
assigned  a  random  value  within  its  phase  space.  This  procedure  produced  very  structured  H20  and 
temperature  profiles  and  provided  a  rigorous  test  of  the  FEOM  algorithm.  Three  sample  H20  and 
temperature  profiles  and  their  resulting  heating  rates  are  shown  in  Figure  6-5. 


Accuracy. 

The  results  displayed  in  the  previous  section  show  that  the  FEOM  prediction  agrees  well  with  the  model 
output.  A  quantitative  assessment  of  the  FEOM  accuracy  was  calculated  by  averaging  the  absolute 
difference  between  the  RTM  calculated  heating  rate  and  the  FEOM  predicted  heating  rate  for  all  1000 
input  sets.  This  was  performed  for  each  atmospheric  layer  represented  in  the  model. 

The  largest  magnitude  average  error  was  3x1  O'9  K/sec.  This  is  103  to  104  times  smaller  than  typical 
heating  rate  calculated  in  the  RTM  and  is  smaller  than  the  error  that  is  incurred  when  the  atmosphere  is 
segmented  into  finite  layers. 
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Table  6-2.  The  number  of  significant  functions  for  each  sub-FEOM  layer. 


Altitude 

level 

Altitude 

(km) 

Number 

of 

1  “  order 
terms 

Number 

of 

2nd  order 
terms 

1 

0.66 

6 

4 

1.33 

8 

7 

1.99 

11 

8 

2.66 

13 

10 

5 

3.33 

13 

11 

6 

3.99 

15 

9 

7 

4.66 

14 

8 

8 

5.33 

12 

9 

9 

6.00 

13 

7 

10 

6.66 

12 

8 

11 

7.33 

12 

7 

12 

8.00 

11 

11 

13 

8.66 

12 

15 

14 

9.33 

14 

7 

15 

10.00 

12 

11 

16 

10.66 

11 

3 

17 

11.33 

10 

1 

18 

12.00 

8 

2 

19 

12.66 

8 

0 

20 

13.33 

7 

0 

21 

14.00 

6 

0 

22 

14.66 

7 

0 

23 

15.33 

8 

0 

24 

16.00 

9 

0 

25 

16.66 

9 

0 

26 

17.33 

9 

0 

27 

18.00 

9 

0 

28 

18.66 

9 

0 

29 

19.33 

9 

0 

30 

20.00 

7 

0 
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Figure  6-5.  Input  H2O  and  temperature  profiles  from  three  of  the  Monte  Carlo  generated  input  sets  (left) 
and  the  corresponding  heating  rate  (right)  calculated  with  the  RTM  (lines)  and  the 
FEOM(symbols). 


235 


Runtime. 


The  runtime  was  assessed  by  timing  the  execution  time  for  RTM  and  FEOM  calculations  for  1000  input 
sets.  The  RTM  required  0.48  seconds  per  calculation  and  the  FEOM  required  0. 1 1  milliseconds  per 
calculation.  Thus  the  FEOM  provides  heating  rate  predictions  103  times  faster  than  the  RTM,  with  the 
same  output  fidelity. 

6. 3. 1.5  Summary  and  Recommendations.  This  section  described  the  successfully  conversion  of  a 
prototype  of  the  Radiative  Transfer  Module  from  RAMS  TO  A  Fully  Equivalent  Operational  Model. 
The  resulting  FEOM  operates  10 3  times  faster  than  the  RTM  while  producing  virtually  identical  output. 
Though  this  proof  of  concept  w'as  limited  to  clear  air  cases,  we  anticipate  similar  quality  results  and  a 
comparable  reduction  in  computational  speed  upon  the  inclusion  of  clouds  and  particles. 

The  demonstrated  ability  to  calculated  heating  rates  10'  times  faster,  while  maintaining  accurate  results, 
could  have  a  tremendous  impact  on  both  RAMS  prediction  fidelity'  and  runtime.  By  completing  the 
FEOM  development  and  replacing  the  existing  RTM  with  a  FEOM,  the  fidelity  o/RAMS  would  be 
improved  through: 


•  Improved  temporal  resolution  of  the  heating  rates,  since  the  FEOM  could  be  called  on  every  time 
step  instead  of  every  10  to  30  steps 

•  Improved  spatial  resolution  of  the  heating  rates  since  the  FEOM  could  be  called  for  every  grid 
column  instead  of  a  small  sample 

Additionally,  it  is  estimated  that  the  overall  runtime  of  RAMS  could  be  reduced  by  up  to  one  third  since 
the  computational  burden  of  the  RTM  would  be  replaced  with  the  ultra-fast  and  accurate  FEOM. 

In  parallel  with  the  RTM  FEOM  development  should  be  the  development  of  a  FEOM  for  the 
microphysics  module.  This  module  is  also  very  computationally  intensive,  and  like  the  RTM,  takes 
about  a  third  of  the  total  RAMS  CPU  time.  There  are  even  cases  when  the  microphysics  module  has 
been  left  out  of  a  RAMS  simulation  to  save  CPU  time.  We  anticipate  that  the  output  of  the 
microphysics  module  could  be  accurately  predicted  with  similar  103  savings  in  computational  time.  The 
impact  of  a  microphysics  FEOM  would  be  both  increased  RAMS  prediction  fidelity  and  reduced  run 
times. 

To  provide  DSWA  with  the  best  tools  to  make  discussions  with,  we  recommend: 

1)  Incorporating  clouds  and  particles  (rain,  snow,  hail,  etc.)  into  the  RTM  FEOM 

2)  Developing  logic  to  piece  together  varying  vertical  resolution  FEOMs  thus  allowing  user 
defined  vertical  layer  structuring 

3)  Implementing  the  completed  RTM  FEOM  into  RAMS 

4)  Testing  of  the  FEOM-  RAMS 

5)  Application  of  FEOM  to  the  cloud  microphysics  module 
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6.3.2  Improvements  To  Numerical  Schemes. 


6. 3.2.1  Exploration  of  a  New  Scheme  for  Computation  of  Gradients.  Determination  of  horizontal 
gradients  in  a  terrain-following  coordinate  system  has  always  been  problematic,  whether  the  vertical 
coordinate  is  height  or  pressure.  All  terrain-following  coordinate  models  have  this  problem.  NCEP's 
ETA  model  was  devised  because  of  the  problem  (although  that  type  of  coordinate  has  it's  own 
limitations).  In  a  signa-z  coordinate  model  such  as  RAMS  ,  the  problem  becomes  most  severe  when  the 
vertical  grid  spacing  is  small  relative  to  the  horizontal  grid  spacing  in  the  presence  of  steep  topography. 
If  the  ratio  of  the  change  in  topography  in  one  grid  space  divided  by  the  minimum  vetical  grid  spacing 
becomes  too  large  (in  the  range  of  4-5),  numerical  truncation  can  occur  which  can  lead  to  non-physical 
results  at  those  grid  pints.  Currently,  in  order  to  rectify  the  error  in  a  model  simulation,  the  following 
methods  can  be  used: 


•  Smooth  the  topography  so  that  the  change  of  height  from  one  grid  space  to  the  next  is  reduced 

•  Increase  the  vertical  grid  spacing 

•  Increase  the  horizontal  grid  resolution 

In  most  instances,  these  methods  are  usually  undesirable.  Therefore,  we  investigated  a  technique  to 
improve  the  accuracy  of  the  horizontal  gradients. 

The  determination  of  a  horizontal  gradient  in  a  traditional  terrain-following  coordinate  system  is  done  by 
looking  at  the  gradient  along  a  ‘sigma’  or  terrain  following  surface,  and  then  decomposing  the  result  into 
vertical  and  horizontal  parts.  This  is  in  contrast  to  taking  a  pure  Cartesian  or  horizontal  gradient  that  is 
determined  by  interpolating  surrounding  values  to  the  same  height.  When  the  vertical  resolution  is 
relatively  coarse,  the  sigma  gradient  and  the  Cartesian  gradient  give  similar  results.  However,  when  the 
vertical  resolution  increases,  the  Cartesian  gradient  in  the  presence  of  steep  topography  crosses  several 
sigma  surfaces  and  this  gradient  then  becomes  different  and  more  accurate  than  a  gradient  determined 
along  a  sigma  surface. 

We  have  been  working  on  a  solution  to  this  problem  by  specifying  horizontal  diffusion  as  a  tendency 
that  results  from  a  3-point  smoothing  operation  along  a  Cartesian  surface.  This  eliminates  the  need  to 
determine  terrain-following  horizontal  gradients  (at  least  in  the  diffusion  scheme)  and  will  allow  for  the 
specification  of  the  vertical  resolution  without  regards  to  the  horizontal  resolution  or  topography 
steepness.  The  new  code  has  been  implemented  into  version  4a  of  RAMS,  but  it  still  needs  additional 
testing  to  see  how  well  it  works  in  a  general  situation.  There  is  also  some  evidence  we  have  recently 
discovered  that  indicates  similar  modifications  may  have  to  be  made  to  the  advective  terms  and  the 
pressure  gradient  term. 
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SECTION  7 
TEST  SUPPORT 


7.1  INTRODUCTION. 

The  Hazard  Assessment  System  for  Consequence  Analysis  (HASCAL)  is  part  of  the  Hazard  Prediction 
and  Assessment  Capability  (HP AC).  The  objective  of  HASCAL  is  to  provide  an  automated  collateral 
effects  prediction  and  tracking  tool  for  weapons  of  mass  destruction  (WMD).  RAMS  forecast  data  is 
one  source  of  meteorological  data  available  to  this  system.  There  were  two  main  focuses  of  our  efforts 
in  support  of  HP  AC.  The  fust  was  to  make  RAMS  easier  to  use  and  implement,  and  the  second  was  to 
integrate  and  test  RAMS’  meteorological  predictions  with  the  hazardous  material  dispersion  code 
(SCIPUFF)  under  development  by  DSWA.  The  meteorological  support  provided  during  these 
demonstration  tests  was  essentially  operational  support  for  the  test  staff.  Little  meteorological  data  was 
available  to  validate  predictions.  Often  the  only  validation  was  matching  visually  observed  plume 
motion  or  deposition  samples  collected  from  the  plume.  This  section  documents  the  support  provided 
and  provides  samples  of  the  predicted  dispersion  patterns. 

To  test  and  integrate  RAMS  with  the  hazardous  material  dispersion  codes,  RAMS  was  used  to  support 
counterproliferation  explosive  testing  at  White  Sands  Missile  Range  (WSMR).  This  testing  was  part  of 
the  overall  Counterproliferation  (CP)  Advanced  Concept  Technology  Demonstration  (ACTD)  of  which 
HP  AC  is  a  key  element.  We  supported  tests  in  the  Dipole  Orbit  (DO),  Dipole  East  (DE)  and  Dipole 
Tiger  (DT)  test  series  (Bradley  et.  al,  1996).  The  model  interfaces  to  the  dispersion  model  SCIPUFF 
(Sykes  et.  al,  1996)  were  refined  and  evaluated  based  on  the  experience  gained  during  the  support  of 
these  explosive  tests.  While  the  objectives  of  these  test  series  varied,  they  all  resulted  in  releases  of 
chemical  or  pseudo-biological  materials  as  a  result  of  an  explosion.  Predicting  the  dispersion  of  these 
materials  in  the  complex  terrain  associated  with  WSMR  was  a  challenging  task. 

While  the  specific  support  plan  for  each  test  varied,  the  general  plan  did  not.  Initially,  relatively  coarse 
(10  km  grid  spacing)  forecasts  were  produced  up  to  72  hours  prior  to  the  expected  event.  This  was 
followed  by  more  finely  resolved  forecasts  (3.3  km  grid  spacing)  for  shorter  periods  prior  to  the  expected 
event.  This  series  of  forecasts  supported  planning  meetings  that  lead  up  to  the  test  event.  In  addition,  a 
very  fine  h  1  km  grid  was  imbedded  in  the  3.3  km  grid  for  post  test  analysis  if  required,  and  for  some 
backup  runs  made  with  an  experimental  parallel  version  of  RAMS.  The  pretest  forecasts  were  usually 
accomplished  on  an  IBM  RS  6000/590  located  at  the  test  site.  Backup  forecasts  were  also  made  on  a 
similar  workstation  located  at  DSWA.  Standard  weather  observations,  local  weather  observations,  and 
regional  forecasts  were  used  to  initialize  the  RAMS  forecasts.  Figure  7-1  depicts  the  weather  data 
network  supporting  the  WSMR  tests.  The  large  round  dots  are  National  Weather  Service  observations 
sites.  The  “plus  signs”  indicate  the  WSMR  surface  observation  system.  These  observations  were  used 
along  with  the  gridded  fields  from  a  regional  National  Center  for  Environmental  Prediction  (NCEP) 
model  (predominately  ETA)  to  produce  the  initial  gridded  fields  RAMS  requires.  The  regional  model 
also  provides  the  time  dependent  boundary  conditions. 

The  nested  grid  capability  in  RAMS  was  used  to  provide  multiple  levels  of  resolution.  The  “2  grid” 
forecasts  used  a  40  km  resolution  grid  with  a  nested  10  km  grid.  The  “3  grid”  nested  an  additional  3.3 
km  resolution  grid  within  the  10  km  grid.  The  grid  structure  for  the  pretest  forecasts  is  shown  in 
Figure  7-2.  Run  times  on  the  IBM  RS  6000/590  were  .13  times  real  time  for  the  “2  grid”  and  .31  time 
real  time  for  the  “3  grid”.  Thus  a  12-hour  “3  grid”  forecast  would  take  about  4  hours  to  complete. 
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Figure  7-2.  Grid  Locations. 


7.2  TEST  SUPPORT. 

As  noted  above  test  support  varied  from  test  to  test  depending  on  whether  a  dry  run  was  scheduled  or  the 
test  was  delayed.  These  circumstances  resulted  in  the  complete  or  partial  repeats  of  the  test  cycle. 
Current  surface  observations  as  well  as  gridded  model  and  forecast  analyses  (ETA  model  for  forecast  up 
to  48  hours  and  MRF  for  forecasts  of  60  or  72  hours)  from  NCEP’s  ftp  server,  were  used  to  initialize 
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RAMS  and  prepare  boundary  conditions  for  forecasts  ranging  in  length  from  12  hours  to  72  hours.  In 
addition,  when  available,  local  observations  from  surface  stations,  a  profiler  network,  and  rawinsondes 
were  included  in  the  initial  analyses.  For  simulations  greater  than  24  hours,  RAMS  was  run  with  2  grids 
The  second  grid  with  grid  spacing  of  10  km  was  nested  within  the  first  grid  with  grid  spacing  of  40  km. 
Simulations  of  24  hours  or  less  allowed  the  use  of  a  further  nested  third  grid  with  a  grid  spacing  of  3.3 
km  centered  over  the  test  site.  Figure  7-3  shows  the  “normal”  experiment  time  line.  Table  7-1  provides 
a  summary  of  the  tests  supported  and  forecasts  produced. 
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Figure  7-3.  Forecast  Timelines. 


Table  7-1.  Demonstration  tests  supported  and  forecasts  produced. 


Experiment 

Initial 

Date  time  fZ] 

Simulation 
period  [h]  - 
No.  grids 

Experiment 

Initial 

Date  time  [Z] 

Simulation 
period  [h]  - 
No.  grids 

DO-1  (3  Dec  Shot) 

Post  test 
predictions  only 

DE-159  (20  July  Shot) 

DO-4  (9  April  Shot) 

5apr00 

24-3  grid 

■Bm 

mmassfsm 

5aprl2 

12-3  grid 

1 9jul00 

WEBE33M 

7apr00 

19jul  12 

60-2  grid 

IHS3H 

24-3  grid 

8aprl6 

9apr00 

DO-3  (10  Dec  Shot) 

9aprl2 

9dec00 

48-2  grid 

lOdecOO 

12-3  grid 
and  48-2  grid 
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Table  7-1.  Demonstration  tests  supported  and  forecasts  produced  (Continued). 


Experiment 

Initial 

Date  time  [Z] 

Simulation 
period  [h] 

No.  grids 

Experiment 

Initial 

Date  time  [Z] 

Simulation 
period  [h] 

No.  grids 

DO-5  (No  Shot) 

DO-6  (6  Feb  Shot) 

sum 

4feb00 

5feb00 

6feb00 

Km 

DO-5b  (11  June  Shot) 

9junl2 

DT-1  (17  April  Shot) 

HHHOHillSHNI 

24-3  grid 

Hardware  error 

36  -  2  grid 

5355 

10junl2 

24-3  grid 

17apr00 

DE-159  (20  July  Shot) 

DT-2  (15  May  Shot) 

flHflDSSESSSH 

72  -  2  grid 

18jul00 

48  -  2  grid 

webesemi 

19iul00 

19jull2 

■ffiSI 

BB9 

7.2.1.  Interface  Development. 

Enhanced  interfaces  were  developed  between  SCIPUFF  and  RAMS.  The  SCIPUFF  model  now  can  run 
with  an  estimate  of  the  surface  heat  flux  and/or  turbulent  kinetic  energy  and  the  boundary  layer  height. 
These  quantities  are  computed  in  RAMS.  As  an  initial  test  of  the  impact  of  using  RAMS  predicted 
fields,  two  dimension  fields  of  surface  heat  flux  and  boundary  layer  height  were  used.  SCIPUFF  was 
run  using  the  predicted  winds  and  its  normal  assumptions,  and  compared  to  the  calculation  using  the 
additional  information.  The  SCIPUFF  predictions  using  the  boundary  layer  information  from  RAMS 
matched  the  observed  test  data  better.  This  was  especially  true  in  cases  like  Dipole  East  159  where  an 
observed  shear  layer  significantly  affected  the  predicted  dispersion  pattern.  The  default  boundary  layer 
height  computed  within  SCIPUFF  was  higher  than  this  layer  causing  the  dispersion  model  to  mix  the 
plume  below  this  height,  effectively  destroying  the  effect  of  the  shear  layer.  The  ability  to  input  the  heat 
flux  and  boundary  layer  height  when  a  sophisticated  model  like  RAMS  is  used  is  now  a  standard 
SCIPUFF  option.  The  results  depicted  below  all  use  this  option.  As  noted  above,  the  initial  transfer  of 
data  from  RAMS  to  SCIPUFF  used  a  gridded  data  field  option  within  SCIPUFF  (the  “MEDOC”  option). 
A  second  method  of  transferring  data  involves  the  use  of  “posts”.  These  “posts”  are  vertical  profiles 
taken  at  a  point  within  the  grid.  SCIPUFF  can  be  used  to  predict  the  dispersion  using  these  data  posts. 
The  use  of  posts  allows  the  meteorological  model  to  be  run  at  one  location  and  the  dispersion  model  at  a 
second  location  with  a  minimum  of  data  transferred.  (A  gridded  data  set  is  actually  an  array  of  posts.) 
The  post  option  in  SCIPUFF  also  allows  the  boundary  layer  and  heat  flux  to  be  specified. 
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7.2.2  Dipole  Orbit  1. 


Dipole  Orbit  1  (DOl)  occurred  in  early  December  1995.  We  did  not  start  providing  meteorological 
support  to  the  HP  AC  program  until  April  1996.  However,  DOl  was  the  first  event  we  computed  using 
RAMS.  We  initially  attempted  to  use  the  data  (surface  observations,  rawinsonde  data,  and  gridded 
analysis)  from  the  National  Center  for  Atmospheric  Research  (NCAR)  archives.  The  resulting  forecasts 
did  not  match  the  observations  at  the  test  site.  Eventually  we  were  able  to  obtain  the  forecast  output  of 
NCEP’s  MRF  model  for  the  test  period  to  replace  the  NCAR  gridded  analysis.  This  resulted  in  wind  and 
dispersion  patterns  that  more  closely  corresponded  to  those  observed. 

The  best  simulation  resulted  from  a  series  of  model  runs.  Initially,  a  24-hour  3-grid  simulation  was  run 
starting  at  OZ  on  3  December  with  a  smallest  grid  spacing  of  3.3  km.  Then,  the  model  was  restarted  at 
12Z  and  a  fourth  grid  with  a  smallest  grid  spacing  of  1.1  km  was  added.  Available  meteorological  was 
assimilated  at  this  time.  Figure  1A  depicts  the  surface  deposition  of  a  simulated  BG  agent  5  hours  after 
the  explosion.  The  terrain  contours  are  shown  in  this  figure.  Mockingbird  Gap  is  located  between  the 
hill  at  top  center  and  the  ridge  is  near  midway  point  on  the  left.  As  can  be  seen,  the  plume  was 
simulated  to  pass  through  the  Mockingbird  Gap  and  continue  on  a  south-southeast  trajectory.  To  better 
reconcile  the  plume  trajectory,  another  four-grid  run  was  re-initialized  at  18Z  with  available 
meteorological  data.  Figure  7-5  shows  the  modified  dispersion  pattern  with  reinitialization.  Only  the 
roads  are  shown  in  this  figure.  This  run  captured  the  southeast  trajectory  of  the  tail  of  the  plume,  but  the 
initial  trajectory  was  too  far  to  the  east.  This  may  be  explained  by  noting  that  SCIPUFF  used  hourly  data 
starting  from  19Z  from  the  re-initialized  run,  so  that  the  trajectory  starting  from  18:45Z  extrapolated  the 
model  fields  from  19Z  and  later  times.  It  is  then  possible  that  if  model  results  were  ingested  into  - 
SCIPUFF  starting  at  shot  time,  the  initial  trajectory  would  be  more  favorable. 

The  SCIPUFF  dispersion  calculation  for  Figure  7-5  was  initiated  using  13  posts  around  the  release  point. 
The  results  reproduce  a  similar  dispersion  pattern  computed  using  the  gridded  MEDOC  format. 


Figure  7-4.  DOl  Four  grid  forecast  at  5  hours. 
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Figure  7-5.  DOl  Four  Grid  Forecast  with  reinitialization 


7.2.3  Dipole  Orbit  4. 

The  first  field  test  supported  at  WSMR  was  Dipole  Orbit  4  (D04),  April  4-9,  1996.  During  this  test, 
we  installed  RAMS  on  DSWA’s  IBM  workstation  model  590.  All  forecasts  were  available  when 
needed.  Data  from  the  3.3  km  fine  grid  were  used  to  successfully  drive  the  SCIPUFF  dispersion  model. 
Good  agreement  between  forecast  and  observation  data  was  obtained  in  most  cases.  Time  dependent 
boundary  conditions  were  obtained  from  NCEP’s  ETA  model  for  up  to  48  hours  and  the  MRF  model  for 
60  and  72  hours.  The  fields  predicted  by  MRF  for  72  hours  did  not  match  those  subsequently  observed. 
They  resulted  in  incorrect  time  dependent  boundary  conditions  that  were  reflected  in  the  RAMS 
predictions.  Shorter  forecasts  made  later  allowed  the  use  of  more  realistic  boundary  conditions.  The 
capability  of  starting  a  RAMS  forecast  from  previous  analysis  files  and  current  WSMR  data  was  also 
demonstrated  during  this  test. 

7.2.4  Dipole  Orbit  6. 

The  test  supported  at  WSMR,  April  22  -  23,  essentially  duplicated  the  first,  except  the  capability  to 
include  the  rawinsonde  data  available  at  WSMR  in  the  analysis  was  included.  This  test  also 
demonstrated  the  ability  to  produce  the  forecast  using  RAMS  remotely.  Once  the  revised  code  had  been 
installed  and  tested,  we  departed  the  site  and  produced  the  required  forecasts  remotely  from  our  office  in 
Fort  Collins,  Colorado. 
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7.2.5  Dipole  Orbit  5  and  5b. 

Nominal  forecast  support  provided  for  the  first  scheduled  event  and  the  actual  test  event. 

7.2.6  Dipole  East  159. 

This  was  the  most  challenging  test  supported.  The  winds  were  relatively  light  and  there  was  a  complex 
flow  pattern  observed  with  a  shear  layer  approximately  6  hours  prior  to  the  test.  Although  there  were 
data  from  balloon  soundings,  surface  mesonet  stations,  mobile  profiler  soundings,  and  lidar,  the  data 
were  not  on  a  consistent  enough  temporal  and  spatial  scale  to  fully  understand  the  flow  fields  during  DE- 
159.  Unfortunately,  this  made  model  verification  difficult.  Although  there  were  few  actual  observations 
against  which  the  code  could  be  checked,  the  plume  tracking  data  compared  to  the  RAMS-SCIPUFF 
predictions  indicated  more  study  was  required.  Figure  7-6  depicts  the  surface  dispersion  of  SF6  5  hours 
after  the  explosion.  Sampler  data  indicate  that  at  least  a  portion  of  the  plume  went  north  and  crossed 
highway  380  (the  horizontal  road  north  of  the  release  point-see  Figure  7-6).  Application  of  the  improved 
interface  that  used  the  RAMS  boundary  layer  height  resulted  in  flow  that  crossed  this  highway.  A 
number  of  parametric  runs  were  made  to  understand  the  reasons  for  the  forecast  results.  The  issue 
concerned  the  correct  surface  moisture.  The  model  had  made  good  predictions  in  the  past  at  this  site,  but 
dry  desert  conditions  were  applicable  for  these  tests.  For  DEI 59,  there  was  rain  in  the  area,  but  not 
necessarily  at  the  test  site,  prior  to  the  release.  These  wet  conditions  can  modify  the  dynamics  and 
thermodynamics  and  might  affect  the  predicted  results.  Sensitivity  simulations  included  allowing  a 
moister  surface  layer,  which  lowered  surface  temperatures,  and  forcing  a  zero  albedo,  which  had  the 
effect  of  minimally  heating  the  lowest  model  level.  Compared  to  the  limited  observations,  it  appears 
that  the  moister  model  run  produced  superior  results,  moving  the  plume  northward  over  Rt.  380. 
Increasing  the  surface  heat  flux  reduced  the  amount  of  northerly  flow  and  reasonable  desert  values 
eliminated  it  entirely. 


Longitude 


Figure  7-6.  DEI 59  dispersion  at  5  hours. 


244 


The  low  level  atmospheric  structure  observed  in  DE  159  brought  into  focus  a  problem  common  to 
models  using  terrain  following  coordinate  systems.  This  was  discussed  in  Section  6.3.2. 1. 

7.2.7  Dipole  Orbit  3. 

The  support  for  Dipole  Orbit  3  was  similar  to  the  support  provided  for  the  other  tests.  A  preliminary 
version  of  the  new  Graphical  User  Interface  (GUI)  was  installed  just  prior  to  this  test  and  user  reaction 
evaluated.  The  GUI  was  determined  to  be  a  great  improvement  over  the  previous  interface. 

7.2.8  Dipole  Orbit  6. 

On  site  support  was  not  provided  for  Dipole  Orbit  (D06).  The  test  staff  at  WSMR  ran  RAMS  using  the 
improved  GUI  for  the  experiment  using  mosdy  two  grids,  while  the  parallel  version  of  RAMS  was  run  at 
MRC  for  three  grids  with  forecasts  up  to  72  hours.  Due  to  the  nature  of  the  synoptic  weather,  it  was 
decided  to  expand  the  RAMS  coarse  grid  to  better  simulate  the  evolution  of  a  large  closed  low  near 
Nevada  that  was  too  close  to  the  northern  and  western  boundary  of  the  original  grid.  Although  the 
evolution  of  the  upper  air  pattern  was  far  superior  in  the  expanded  grid,  simulated  near-surface 
conditions  were  similar  to  those  simulated  with  the  smaller  grid. 

7.2.9  Dipole  Tiger  1. 

Support  for  Dipole  Tiger  1  (DT-1)  was  provided  by  the  DSWA  test  staff  at  WSMR  with  backup 
forecasts  on  the  workstation  at  DSWA.  A  timely  24  hour  forecast  was  not  produced  at  the  test  site  due 
to  a  computer  malfunction  that  occurred  after  forecast  initiation  and  the  staffs  departure  for  the  night. 
The  calculation  was  restarted  the  next  morning  after  the  problem  had  been  corrected  but  insufficient  time 
remained  prior  to  the  test  event  to  produce  the  forecast.  Forecast  results  were  good. 

7.2.10  Dipole  Tiger  2. 

The  planned  support  configuration  was  the  same  as  DT-1  with  the  primary  forecast  produce  by  the  staff 
at  WSMR  and  backup  forecast  support  from  the  workstation  at  DSWA.  The  improved  GUI  allowed  the 
DSWA  WSMR  test  staff  to  run  the  forecast  remotely,  demonstrating  its  ease  of  use. 
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